2026-02-17T05:27:56.604 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-17T05:27:56.619 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T05:27:56.722 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51019 branch: wip-bharath19-testing-2026-02-13-0903-squid description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '51019' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath19-testing-2026-02-13-0903-squid ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e workunit: branch: wip-bharath19-testing-2026-02-13-0903-squid sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 3603 sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath19-testing-2026-02-13-0903-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e targets: trial074.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMGX8SVlRPpZgz8ZhCqyKmyqGk47XmnWk1OUeJccUvmNaiFegZCAD0ct3cPT6+b4NcpJS7bgyqP9qPpaZeChz5M= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-17_04:03:08 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T05:27:56.722 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa; will attempt to use it 2026-02-17T05:27:56.722 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/qa/tasks 2026-02-17T05:27:56.722 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T05:27:56.724 INFO:teuthology.task.internal:Checking packages... 2026-02-17T05:27:56.724 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e' 2026-02-17T05:27:56.724 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T05:27:56.724 INFO:teuthology.packaging:ref: None 2026-02-17T05:27:56.724 INFO:teuthology.packaging:tag: None 2026-02-17T05:27:56.724 INFO:teuthology.packaging:branch: wip-bharath19-testing-2026-02-13-0903-squid 2026-02-17T05:27:56.724 INFO:teuthology.packaging:sha1: 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:27:56.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath19-testing-2026-02-13-0903-squid 2026-02-17T05:27:56.864 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-705-g04cf6dd4-1jammy 2026-02-17T05:27:56.865 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T05:27:56.880 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T05:27:56.880 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T05:27:56.896 INFO:teuthology.task.internal:Saving configuration 2026-02-17T05:27:56.905 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T05:27:56.918 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T05:27:57.004 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial074.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51019', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 05:24:34.562137', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILnO/MKPlA5/fjL/73v9zQv01UXmIJrDj9KWQ50pjx0T'} 2026-02-17T05:27:57.004 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T05:27:57.015 INFO:teuthology.task.internal:roles: ubuntu@trial074.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T05:27:57.016 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T05:27:57.104 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x79b421af8040>, signals=[15]) 2026-02-17T05:27:57.104 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T05:27:57.115 INFO:teuthology.task.internal:Opening connections... 2026-02-17T05:27:57.115 DEBUG:teuthology.task.internal:connecting to ubuntu@trial074.front.sepia.ceph.com 2026-02-17T05:27:57.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:27:57.187 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T05:27:57.198 DEBUG:teuthology.orchestra.run.trial074:> uname -m 2026-02-17T05:27:57.266 INFO:teuthology.orchestra.run.trial074.stdout:x86_64 2026-02-17T05:27:57.266 DEBUG:teuthology.orchestra.run.trial074:> cat /etc/os-release 2026-02-17T05:27:57.312 INFO:teuthology.orchestra.run.trial074.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-17T05:27:57.312 INFO:teuthology.orchestra.run.trial074.stdout:NAME="Ubuntu" 2026-02-17T05:27:57.312 INFO:teuthology.orchestra.run.trial074.stdout:VERSION_ID="22.04" 2026-02-17T05:27:57.312 INFO:teuthology.orchestra.run.trial074.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:VERSION_CODENAME=jammy 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:ID=ubuntu 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:ID_LIKE=debian 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-17T05:27:57.313 INFO:teuthology.orchestra.run.trial074.stdout:UBUNTU_CODENAME=jammy 2026-02-17T05:27:57.313 INFO:teuthology.lock.ops:Updating trial074.front.sepia.ceph.com on lock server 2026-02-17T05:27:57.393 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T05:27:57.406 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T05:27:57.417 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T05:27:57.417 DEBUG:teuthology.orchestra.run.trial074:> test '!' -e /home/ubuntu/cephtest 2026-02-17T05:27:57.420 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T05:27:57.430 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T05:27:57.430 DEBUG:teuthology.orchestra.run.trial074:> test -z $(ls -A /var/lib/ceph) 2026-02-17T05:27:57.464 INFO:teuthology.orchestra.run.trial074.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-17T05:27:57.465 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T05:27:57.544 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T05:27:57.564 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-17T05:27:57.565 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-17T05:27:57.565 DEBUG:teuthology.orchestra.run.trial074:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T05:27:57.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:27:57.568 DEBUG:teuthology.orchestra.run.trial074:> uname -r 2026-02-17T05:27:57.612 INFO:teuthology.orchestra.run.trial074.stdout:5.15.0-164-generic 2026-02-17T05:27:57.613 INFO:teuthology.task.kernel:Running kernel on trial074: 5.15.0-164-generic 2026-02-17T05:27:57.613 DEBUG:teuthology.orchestra.run.trial074:> sudo apt-get clean 2026-02-17T05:27:57.671 DEBUG:teuthology.orchestra.run.trial074:> sudo apt-get update 2026-02-17T05:27:57.868 INFO:teuthology.orchestra.run.trial074.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T05:27:57.875 INFO:teuthology.orchestra.run.trial074.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-17T05:27:57.913 INFO:teuthology.orchestra.run.trial074.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-17T05:27:57.929 INFO:teuthology.orchestra.run.trial074.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-17T05:27:58.004 INFO:teuthology.orchestra.run.trial074.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3237 kB] 2026-02-17T05:27:58.070 INFO:teuthology.orchestra.run.trial074.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [493 kB] 2026-02-17T05:27:58.076 INFO:teuthology.orchestra.run.trial074.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-17T05:27:58.076 INFO:teuthology.orchestra.run.trial074.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1255 kB] 2026-02-17T05:27:58.083 INFO:teuthology.orchestra.run.trial074.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [313 kB] 2026-02-17T05:27:58.085 INFO:teuthology.orchestra.run.trial074.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-17T05:27:58.086 INFO:teuthology.orchestra.run.trial074.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5287 kB] 2026-02-17T05:27:58.110 INFO:teuthology.orchestra.run.trial074.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [997 kB] 2026-02-17T05:27:58.115 INFO:teuthology.orchestra.run.trial074.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2966 kB] 2026-02-17T05:27:58.128 INFO:teuthology.orchestra.run.trial074.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [426 kB] 2026-02-17T05:27:58.129 INFO:teuthology.orchestra.run.trial074.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-17T05:27:58.129 INFO:teuthology.orchestra.run.trial074.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1017 kB] 2026-02-17T05:27:58.143 INFO:teuthology.orchestra.run.trial074.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [224 kB] 2026-02-17T05:27:58.144 INFO:teuthology.orchestra.run.trial074.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.6 kB] 2026-02-17T05:27:58.144 INFO:teuthology.orchestra.run.trial074.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5096 kB] 2026-02-17T05:27:58.172 INFO:teuthology.orchestra.run.trial074.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [964 kB] 2026-02-17T05:28:01.859 INFO:teuthology.orchestra.run.trial074.stdout:Fetched 22.7 MB in 2s (13.5 MB/s) 2026-02-17T05:28:02.396 INFO:teuthology.orchestra.run.trial074.stdout:Reading package lists... 2026-02-17T05:28:02.410 DEBUG:teuthology.orchestra.run.trial074:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T05:28:02.442 INFO:teuthology.orchestra.run.trial074.stdout:Reading package lists... 2026-02-17T05:28:02.551 INFO:teuthology.orchestra.run.trial074.stdout:Building dependency tree... 2026-02-17T05:28:02.552 INFO:teuthology.orchestra.run.trial074.stdout:Reading state information... 2026-02-17T05:28:02.639 INFO:teuthology.orchestra.run.trial074.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T05:28:02.639 INFO:teuthology.orchestra.run.trial074.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T05:28:02.639 INFO:teuthology.orchestra.run.trial074.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T05:28:02.639 INFO:teuthology.orchestra.run.trial074.stdout: sg3-utils-udev 2026-02-17T05:28:02.639 INFO:teuthology.orchestra.run.trial074.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T05:28:02.639 INFO:teuthology.orchestra.run.trial074.stdout:The following additional packages will be installed: 2026-02-17T05:28:02.639 INFO:teuthology.orchestra.run.trial074.stdout: linux-generic linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T05:28:02.640 INFO:teuthology.orchestra.run.trial074.stdout: linux-headers-generic linux-image-5.15.0-170-generic 2026-02-17T05:28:02.640 INFO:teuthology.orchestra.run.trial074.stdout: linux-modules-5.15.0-170-generic linux-modules-extra-5.15.0-170-generic 2026-02-17T05:28:02.640 INFO:teuthology.orchestra.run.trial074.stdout:Suggested packages: 2026-02-17T05:28:02.640 INFO:teuthology.orchestra.run.trial074.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-17T05:28:02.652 INFO:teuthology.orchestra.run.trial074.stdout:The following NEW packages will be installed: 2026-02-17T05:28:02.652 INFO:teuthology.orchestra.run.trial074.stdout: linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T05:28:02.652 INFO:teuthology.orchestra.run.trial074.stdout: linux-image-5.15.0-170-generic linux-modules-5.15.0-170-generic 2026-02-17T05:28:02.652 INFO:teuthology.orchestra.run.trial074.stdout: linux-modules-extra-5.15.0-170-generic 2026-02-17T05:28:02.652 INFO:teuthology.orchestra.run.trial074.stdout:The following packages will be upgraded: 2026-02-17T05:28:02.652 INFO:teuthology.orchestra.run.trial074.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-17T05:28:03.075 INFO:teuthology.orchestra.run.trial074.stdout:3 upgraded, 5 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T05:28:03.075 INFO:teuthology.orchestra.run.trial074.stdout:Need to get 113 MB of archives. 2026-02-17T05:28:03.076 INFO:teuthology.orchestra.run.trial074.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-17T05:28:03.076 INFO:teuthology.orchestra.run.trial074.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-170-generic amd64 5.15.0-170.180 [22.7 MB] 2026-02-17T05:28:04.460 INFO:teuthology.orchestra.run.trial074.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-170-generic amd64 5.15.0-170.180 [11.6 MB] 2026-02-17T05:28:04.801 INFO:teuthology.orchestra.run.trial074.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-170-generic amd64 5.15.0-170.180 [64.0 MB] 2026-02-17T05:28:06.689 INFO:teuthology.orchestra.run.trial074.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.170.159 [1654 B] 2026-02-17T05:28:06.689 INFO:teuthology.orchestra.run.trial074.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.170.159 [2474 B] 2026-02-17T05:28:06.690 INFO:teuthology.orchestra.run.trial074.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170 all 5.15.0-170.180 [12.4 MB] 2026-02-17T05:28:07.044 INFO:teuthology.orchestra.run.trial074.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170-generic amd64 5.15.0-170.180 [2828 kB] 2026-02-17T05:28:07.125 INFO:teuthology.orchestra.run.trial074.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.170.159 [2316 B] 2026-02-17T05:28:07.323 INFO:teuthology.orchestra.run.trial074.stdout:Fetched 113 MB in 4s (25.4 MB/s) 2026-02-17T05:28:07.343 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package linux-modules-5.15.0-170-generic. 2026-02-17T05:28:07.506 INFO:teuthology.orchestra.run.trial074.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 131954 files and directories currently installed.) 2026-02-17T05:28:07.507 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../0-linux-modules-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T05:28:07.508 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:07.854 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package linux-image-5.15.0-170-generic. 2026-02-17T05:28:07.860 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../1-linux-image-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T05:28:07.867 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:07.909 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-170-generic. 2026-02-17T05:28:07.916 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T05:28:07.916 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:08.999 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../3-linux-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T05:28:09.000 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T05:28:09.021 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T05:28:09.022 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-image-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T05:28:09.037 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package linux-headers-5.15.0-170. 2026-02-17T05:28:09.044 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../5-linux-headers-5.15.0-170_5.15.0-170.180_all.deb ... 2026-02-17T05:28:09.045 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T05:28:10.435 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package linux-headers-5.15.0-170-generic. 2026-02-17T05:28:10.447 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../6-linux-headers-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T05:28:10.448 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:11.027 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T05:28:11.028 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking linux-headers-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T05:28:11.053 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T05:28:11.055 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:11.056 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-headers-generic (5.15.0.170.159) ... 2026-02-17T05:28:11.057 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:11.825 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:12.663 INFO:teuthology.orchestra.run.trial074.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-17T05:28:12.663 INFO:teuthology.orchestra.run.trial074.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-170-generic 2026-02-17T05:28:12.669 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:13.463 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-image-generic (5.15.0.170.159) ... 2026-02-17T05:28:13.464 INFO:teuthology.orchestra.run.trial074.stdout:Setting up linux-generic (5.15.0.170.159) ... 2026-02-17T05:28:13.466 INFO:teuthology.orchestra.run.trial074.stdout:Processing triggers for linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T05:28:13.470 INFO:teuthology.orchestra.run.trial074.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-17T05:28:13.471 INFO:teuthology.orchestra.run.trial074.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-170-generic 2026-02-17T05:28:20.509 INFO:teuthology.orchestra.run.trial074.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-17T05:28:27.076 INFO:teuthology.orchestra.run.trial074.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-17T05:28:27.077 INFO:teuthology.orchestra.run.trial074.stdout:Sourcing file `/etc/default/grub' 2026-02-17T05:28:27.095 INFO:teuthology.orchestra.run.trial074.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T05:28:27.096 INFO:teuthology.orchestra.run.trial074.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T05:28:27.098 INFO:teuthology.orchestra.run.trial074.stdout:Generating grub configuration file ... 2026-02-17T05:28:27.177 INFO:teuthology.orchestra.run.trial074.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T05:28:27.183 INFO:teuthology.orchestra.run.trial074.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T05:28:27.352 INFO:teuthology.orchestra.run.trial074.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T05:28:27.358 INFO:teuthology.orchestra.run.trial074.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T05:28:27.428 INFO:teuthology.orchestra.run.trial074.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T05:28:27.428 INFO:teuthology.orchestra.run.trial074.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T05:28:27.429 INFO:teuthology.orchestra.run.trial074.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T05:28:27.430 INFO:teuthology.orchestra.run.trial074.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:28:27.440 INFO:teuthology.orchestra.run.trial074.stdout:done 2026-02-17T05:28:27.847 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.847 INFO:teuthology.orchestra.run.trial074.stdout:Pending kernel upgrade! 2026-02-17T05:28:27.847 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.847 INFO:teuthology.orchestra.run.trial074.stdout:Running kernel version: 2026-02-17T05:28:27.847 INFO:teuthology.orchestra.run.trial074.stdout: 5.15.0-164-generic 2026-02-17T05:28:27.847 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.847 INFO:teuthology.orchestra.run.trial074.stdout:Diagnostics: 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout: The currently running kernel version is not the expected kernel version 5.15.0-170-generic. 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout:The processor microcode seems to be up-to-date. 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout:No services need to be restarted. 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout:No containers need to be restarted. 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout:No user sessions are running outdated binaries. 2026-02-17T05:28:27.848 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:28:27.849 INFO:teuthology.orchestra.run.trial074.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T05:28:28.496 DEBUG:teuthology.orchestra.run.trial074:> dpkg -s linux-image-generic 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Package: linux-image-generic 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Status: install ok installed 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Priority: optional 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Section: kernel 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Installed-Size: 21 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Architecture: amd64 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Source: linux-meta 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Version: 5.15.0.170.159 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Recommends: thermald 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout:Description: Generic Linux kernel image 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout: This package will always depend on the latest generic kernel image 2026-02-17T05:28:28.547 INFO:teuthology.orchestra.run.trial074.stdout: available. 2026-02-17T05:28:28.548 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-164-generic Expected: 5.15.0-170-generic 2026-02-17T05:28:28.548 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T05:28:28.548 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T05:28:28.548 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T05:28:28.548 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial074.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T05:28:28.548 DEBUG:teuthology.orchestra.run.trial074:> sudo apt-get clean 2026-02-17T05:28:28.630 DEBUG:teuthology.orchestra.run.trial074:> sudo apt-get update 2026-02-17T05:28:29.124 INFO:teuthology.orchestra.run.trial074.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T05:28:29.206 INFO:teuthology.orchestra.run.trial074.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T05:28:29.295 INFO:teuthology.orchestra.run.trial074.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T05:28:29.383 INFO:teuthology.orchestra.run.trial074.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T05:28:30.025 INFO:teuthology.orchestra.run.trial074.stdout:Reading package lists... 2026-02-17T05:28:30.040 DEBUG:teuthology.orchestra.run.trial074:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T05:28:30.073 INFO:teuthology.orchestra.run.trial074.stdout:Reading package lists... 2026-02-17T05:28:30.183 INFO:teuthology.orchestra.run.trial074.stdout:Building dependency tree... 2026-02-17T05:28:30.184 INFO:teuthology.orchestra.run.trial074.stdout:Reading state information... 2026-02-17T05:28:30.269 INFO:teuthology.orchestra.run.trial074.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-17T05:28:30.269 INFO:teuthology.orchestra.run.trial074.stdout:linux-image-generic set to manually installed. 2026-02-17T05:28:30.269 INFO:teuthology.orchestra.run.trial074.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T05:28:30.269 INFO:teuthology.orchestra.run.trial074.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T05:28:30.269 INFO:teuthology.orchestra.run.trial074.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T05:28:30.270 INFO:teuthology.orchestra.run.trial074.stdout: sg3-utils-udev 2026-02-17T05:28:30.270 INFO:teuthology.orchestra.run.trial074.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T05:28:30.289 INFO:teuthology.orchestra.run.trial074.stdout:0 upgraded, 0 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T05:28:30.290 DEBUG:teuthology.orchestra.run.trial074:> dpkg -s linux-image-generic 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Package: linux-image-generic 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Status: install ok installed 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Priority: optional 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Section: kernel 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Installed-Size: 21 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Architecture: amd64 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Source: linux-meta 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Version: 5.15.0.170.159 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Recommends: thermald 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout:Description: Generic Linux kernel image 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout: This package will always depend on the latest generic kernel image 2026-02-17T05:28:30.343 INFO:teuthology.orchestra.run.trial074.stdout: available. 2026-02-17T05:28:30.344 DEBUG:teuthology.orchestra.run.trial074:> mktemp 2026-02-17T05:28:30.390 INFO:teuthology.orchestra.run.trial074.stdout:/tmp/tmp.WTQb6RWm2Y 2026-02-17T05:28:30.391 DEBUG:teuthology.orchestra.run.trial074:> sudo cp /boot/grub/grub.cfg /tmp/tmp.WTQb6RWm2Y 2026-02-17T05:28:30.442 DEBUG:teuthology.orchestra.run.trial074:> sudo chmod 0666 /tmp/tmp.WTQb6RWm2Y 2026-02-17T05:28:30.536 DEBUG:teuthology.orchestra.remote:trial074:/tmp/tmp.WTQb6RWm2Y is 7KB 2026-02-17T05:28:30.544 DEBUG:teuthology.orchestra.run.trial074:> rm -fr /tmp/tmp.WTQb6RWm2Y 2026-02-17T05:28:30.549 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-17T05:28:30.603 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:28:30.603 DEBUG:teuthology.orchestra.run.trial074:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-17T05:28:30.603 DEBUG:teuthology.orchestra.run.trial074:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-17T05:28:30.664 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-170-generic 2026-02-17T05:28:30.665 DEBUG:teuthology.orchestra.run.trial074:> sudo update-grub 2026-02-17T05:28:30.899 INFO:teuthology.orchestra.run.trial074.stderr:Sourcing file `/etc/default/grub' 2026-02-17T05:28:30.918 INFO:teuthology.orchestra.run.trial074.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T05:28:30.918 INFO:teuthology.orchestra.run.trial074.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T05:28:30.920 INFO:teuthology.orchestra.run.trial074.stderr:Generating grub configuration file ... 2026-02-17T05:28:31.001 INFO:teuthology.orchestra.run.trial074.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T05:28:31.006 INFO:teuthology.orchestra.run.trial074.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T05:28:31.178 INFO:teuthology.orchestra.run.trial074.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T05:28:31.183 INFO:teuthology.orchestra.run.trial074.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T05:28:31.253 INFO:teuthology.orchestra.run.trial074.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T05:28:31.253 INFO:teuthology.orchestra.run.trial074.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T05:28:31.253 INFO:teuthology.orchestra.run.trial074.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T05:28:31.255 INFO:teuthology.orchestra.run.trial074.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T05:28:31.262 INFO:teuthology.orchestra.run.trial074.stderr:done 2026-02-17T05:28:31.264 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial074 2026-02-17T05:28:31.264 DEBUG:teuthology.orchestra.run.trial074:> sudo bash -c ' 2026-02-17T05:28:31.264 DEBUG:teuthology.orchestra.run.trial074:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T05:28:31.264 DEBUG:teuthology.orchestra.run.trial074:> ' 2026-02-17T05:29:01.267 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T05:29:01.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial074.front.sepia.ceph.com' 2026-02-17T05:29:01.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:30:01.325 DEBUG:teuthology.orchestra.remote:timed out 2026-02-17T05:30:10.332 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial074.front.sepia.ceph.com' 2026-02-17T05:30:10.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:30:10.461 DEBUG:teuthology.orchestra.run.trial074:> true 2026-02-17T05:30:10.857 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial074.front.sepia.ceph.com' 2026-02-17T05:30:10.857 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-170-generic"... 2026-02-17T05:30:10.857 DEBUG:teuthology.orchestra.run.trial074:> uname -r 2026-02-17T05:30:10.901 INFO:teuthology.orchestra.run.trial074.stdout:5.15.0-170-generic 2026-02-17T05:30:10.902 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-170-generic vs 5.15.0-170-generic 2026-02-17T05:30:10.902 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T05:30:10.902 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-17T05:30:11.902 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T05:30:11.903 DEBUG:teuthology.orchestra.run.trial074:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T05:30:11.914 INFO:teuthology.orchestra.run.trial074.stdout:ttyS1 2026-02-17T05:30:11.931 DEBUG:teuthology.parallel:result is None 2026-02-17T05:30:11.931 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T05:30:11.941 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T05:30:11.942 DEBUG:teuthology.orchestra.run.trial074:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T05:30:11.976 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T05:30:11.987 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T05:30:12.003 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T05:30:12.004 DEBUG:teuthology.orchestra.run.trial074:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T05:30:12.026 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T05:30:12.038 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T05:30:12.038 DEBUG:teuthology.orchestra.run.trial074:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T05:30:12.070 INFO:teuthology.orchestra.run.trial074.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:30:12.076 INFO:teuthology.orchestra.run.trial074.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:30:12.077 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T05:30:12.088 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T05:30:12.088 DEBUG:teuthology.orchestra.run.trial074:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T05:30:12.129 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T05:30:12.141 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T05:30:12.141 DEBUG:teuthology.orchestra.run.trial074:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T05:30:12.173 DEBUG:teuthology.orchestra.run.trial074:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T05:30:12.218 DEBUG:teuthology.orchestra.run.trial074:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T05:30:12.266 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:30:12.266 DEBUG:teuthology.orchestra.run.trial074:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T05:30:12.317 DEBUG:teuthology.orchestra.run.trial074:> sudo service rsyslog restart 2026-02-17T05:30:12.374 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T05:30:12.387 INFO:teuthology.task.internal:Starting timer... 2026-02-17T05:30:12.387 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T05:30:12.409 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T05:30:12.426 DEBUG:teuthology.task.selinux:Excluding trial074: OS 'ubuntu' does not support SELinux 2026-02-17T05:30:12.426 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T05:30:12.426 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-17T05:30:12.426 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T05:30:12.427 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T05:30:12.443 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-17T05:30:12.447 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-17T05:30:12.607 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T05:30:12.618 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T05:30:12.619 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial074.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T05:32:33.353 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial074.front.sepia.ceph.com')] 2026-02-17T05:32:33.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial074.front.sepia.ceph.com' 2026-02-17T05:32:33.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T05:32:33.466 DEBUG:teuthology.orchestra.run.trial074:> true 2026-02-17T05:32:33.712 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial074.front.sepia.ceph.com' 2026-02-17T05:32:33.712 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T05:32:33.736 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T05:32:33.737 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T05:32:33.737 DEBUG:teuthology.orchestra.run.trial074:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Command line: ntpd -gq 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: ---------------------------------------------------- 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: ntp-4 is maintained by Network Time Foundation, 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: corporation. Support and training for ntp-4 are 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: available at https://www.nwtime.org/support 2026-02-17T05:32:33.774 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: ---------------------------------------------------- 2026-02-17T05:32:33.775 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: proto: precision = 0.030 usec (-25) 2026-02-17T05:32:33.775 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: basedate set to 2022-02-04 2026-02-17T05:32:33.775 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: gps base set to 2022-02-06 (week 2196) 2026-02-17T05:32:33.775 INFO:teuthology.orchestra.run.trial074.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T05:32:33.776 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T05:32:33.776 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: restrict ::: KOD does nothing without LIMITED. 2026-02-17T05:32:33.776 INFO:teuthology.orchestra.run.trial074.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-17T05:32:33.777 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Listen and drop on 0 v6wildcard [::]:123 2026-02-17T05:32:33.777 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-17T05:32:33.777 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Listen normally on 2 lo 127.0.0.1:123 2026-02-17T05:32:33.777 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Listen normally on 3 enp1s0f0 10.20.193.74:123 2026-02-17T05:32:33.777 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Listen normally on 4 lo [::1]:123 2026-02-17T05:32:33.777 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:60a0%2]:123 2026-02-17T05:32:33.778 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:33 ntpd[8775]: Listening on routing socket on fd #22 for interface updates 2026-02-17T05:32:34.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:34.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:34.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:34.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:34.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:34.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:34.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:34.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:34.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:34.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:34.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:34.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:34 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:35.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:35.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:35.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:35.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:35.783 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:35.783 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:35.783 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:35.783 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:35 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:35.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:35.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:35.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:35.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:36.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:36.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:36.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:36.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:36.777 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:36.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:36.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:36.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:36.796 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:36.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:36.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:36.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:36 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:37.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:37.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:37.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:37.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:37.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:37.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:37.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:37.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:37 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:37.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:37.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:37.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:37.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:38.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:38.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:38.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:38.775 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:38.777 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:38.777 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:38.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:38.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:38.796 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:38.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:38.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:38.797 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:38 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:39.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:39.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:39.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:39.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:39.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:39.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:39.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:39.782 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:39 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:39.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:39.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:39.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:39.804 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:40.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:40.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:40.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:40.774 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:40.777 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T05:32:40.777 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T05:32:40.777 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T05:32:40.777 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T05:32:40.777 INFO:teuthology.orchestra.run.trial074.stdout:17 Feb 05:32:40 ntpd[8775]: ntpd: time set -0.204487 s 2026-02-17T05:32:40.777 INFO:teuthology.orchestra.run.trial074.stdout:ntpd: time set -0.204487s 2026-02-17T05:32:40.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-17T05:32:40.778 INFO:teuthology.orchestra.run.trial074.stderr:17 Feb 05:32:40 ntpd[8775]: can't open /var/log/ntpstats/loopstats.20260217: Permission denied 2026-02-17T05:32:41.673 INFO:teuthology.orchestra.run.trial074.stdout: remote refid st t when poll reach delay offset jitter 2026-02-17T05:32:41.673 INFO:teuthology.orchestra.run.trial074.stdout:============================================================================== 2026-02-17T05:32:41.674 INFO:teuthology.orchestra.run.trial074.stdout: 23.186.168.132 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T05:32:41.674 INFO:teuthology.orchestra.run.trial074.stdout: ntp1.wiktel.com .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T05:32:41.674 INFO:teuthology.orchestra.run.trial074.stdout: ntp.netlinkify. .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T05:32:41.674 INFO:teuthology.orchestra.run.trial074.stdout: 204.2.134.173 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T05:32:41.675 INFO:teuthology.run_tasks:Running task install... 2026-02-17T05:32:41.688 DEBUG:teuthology.task.install:project ceph 2026-02-17T05:32:41.688 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'}} 2026-02-17T05:32:41.688 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'} 2026-02-17T05:32:41.689 INFO:teuthology.task.install:Using flavor: default 2026-02-17T05:32:41.698 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T05:32:41.698 INFO:teuthology.task.install:extra packages: [] 2026-02-17T05:32:41.698 DEBUG:teuthology.orchestra.run.trial074:> sudo apt-key list | grep Ceph 2026-02-17T05:32:41.771 INFO:teuthology.orchestra.run.trial074.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-17T05:32:41.800 INFO:teuthology.orchestra.run.trial074.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-17T05:32:41.800 INFO:teuthology.orchestra.run.trial074.stdout:uid [ unknown] Ceph.com (release key) 2026-02-17T05:32:41.801 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-17T05:32:41.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:32:41.950 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default/ 2026-02-17T05:32:41.950 INFO:teuthology.task.install.deb:Package version is 19.2.3-705-g04cf6dd4-1jammy 2026-02-17T05:32:42.086 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:32:42.086 DEBUG:teuthology.orchestra.run.trial074:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-17T05:32:42.096 DEBUG:teuthology.orchestra.run.trial074:> sudo apt-get update 2026-02-17T05:32:42.245 INFO:teuthology.orchestra.run.trial074.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy InRelease 2026-02-17T05:32:42.277 INFO:teuthology.orchestra.run.trial074.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy Release [7662 B] 2026-02-17T05:32:42.289 INFO:teuthology.orchestra.run.trial074.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-17T05:32:42.333 INFO:teuthology.orchestra.run.trial074.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2026-02-17T05:32:42.586 INFO:teuthology.orchestra.run.trial074.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T05:32:42.669 INFO:teuthology.orchestra.run.trial074.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T05:32:42.757 INFO:teuthology.orchestra.run.trial074.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T05:32:42.846 INFO:teuthology.orchestra.run.trial074.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T05:32:42.993 INFO:teuthology.orchestra.run.trial074.stdout:Fetched 25.8 kB in 1s (34.9 kB/s) 2026-02-17T05:32:43.506 INFO:teuthology.orchestra.run.trial074.stdout:Reading package lists... 2026-02-17T05:32:43.520 DEBUG:teuthology.orchestra.run.trial074:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.2.3-705-g04cf6dd4-1jammy cephadm=19.2.3-705-g04cf6dd4-1jammy ceph-mds=19.2.3-705-g04cf6dd4-1jammy ceph-mgr=19.2.3-705-g04cf6dd4-1jammy ceph-common=19.2.3-705-g04cf6dd4-1jammy ceph-fuse=19.2.3-705-g04cf6dd4-1jammy ceph-test=19.2.3-705-g04cf6dd4-1jammy ceph-volume=19.2.3-705-g04cf6dd4-1jammy radosgw=19.2.3-705-g04cf6dd4-1jammy python3-rados=19.2.3-705-g04cf6dd4-1jammy python3-rgw=19.2.3-705-g04cf6dd4-1jammy python3-cephfs=19.2.3-705-g04cf6dd4-1jammy python3-rbd=19.2.3-705-g04cf6dd4-1jammy libcephfs2=19.2.3-705-g04cf6dd4-1jammy libcephfs-dev=19.2.3-705-g04cf6dd4-1jammy librados2=19.2.3-705-g04cf6dd4-1jammy librbd1=19.2.3-705-g04cf6dd4-1jammy rbd-fuse=19.2.3-705-g04cf6dd4-1jammy 2026-02-17T05:32:43.554 INFO:teuthology.orchestra.run.trial074.stdout:Reading package lists... 2026-02-17T05:32:43.662 INFO:teuthology.orchestra.run.trial074.stdout:Building dependency tree... 2026-02-17T05:32:43.662 INFO:teuthology.orchestra.run.trial074.stdout:Reading state information... 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout: sg3-utils-udev 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout:The following additional packages will be installed: 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-17T05:32:43.753 INFO:teuthology.orchestra.run.trial074.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: pkg-config python-asyncssh-doc python-pastedeploy-tpl python3-asyncssh 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cheroot 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-17T05:32:43.754 INFO:teuthology.orchestra.run.trial074.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python3-pyinotify python3-pytest python3-repoze.lru 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python3-simplejson python3-singledispatch python3-sklearn 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python3-toml python3-waitress python3-wcwidth python3-webob 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout:Suggested packages: 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python3-influxdb readline-doc python3-beaker python-mako-doc 2026-02-17T05:32:43.755 INFO:teuthology.orchestra.run.trial074.stdout: python-natsort-doc httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2026-02-17T05:32:43.756 INFO:teuthology.orchestra.run.trial074.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2026-02-17T05:32:43.756 INFO:teuthology.orchestra.run.trial074.stdout: python-pygments-doc ttf-bitstream-vera python-pyinotify-doc python3-dap 2026-02-17T05:32:43.756 INFO:teuthology.orchestra.run.trial074.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2026-02-17T05:32:43.756 INFO:teuthology.orchestra.run.trial074.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2026-02-17T05:32:43.756 INFO:teuthology.orchestra.run.trial074.stdout:Recommended packages: 2026-02-17T05:32:43.756 INFO:teuthology.orchestra.run.trial074.stdout: btrfs-tools 2026-02-17T05:32:43.787 INFO:teuthology.orchestra.run.trial074.stdout:The following NEW packages will be installed: 2026-02-17T05:32:43.787 INFO:teuthology.orchestra.run.trial074.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-17T05:32:43.787 INFO:teuthology.orchestra.run.trial074.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: python-pastedeploy-tpl python3-asyncssh python3-cachetools 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cheroot 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-17T05:32:43.788 INFO:teuthology.orchestra.run.trial074.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-threadpoolctl python3-toml python3-waitress python3-wcwidth 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse socat unzip 2026-02-17T05:32:43.789 INFO:teuthology.orchestra.run.trial074.stdout: xmlstarlet zip 2026-02-17T05:32:43.895 INFO:teuthology.orchestra.run.trial074.stdout:0 upgraded, 108 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T05:32:43.895 INFO:teuthology.orchestra.run.trial074.stdout:Need to get 178 MB of archives. 2026-02-17T05:32:43.895 INFO:teuthology.orchestra.run.trial074.stdout:After this operation, 809 MB of additional disk space will be used. 2026-02-17T05:32:43.895 INFO:teuthology.orchestra.run.trial074.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.2.3-705-g04cf6dd4-1jammy [3593 kB] 2026-02-17T05:32:44.033 INFO:teuthology.orchestra.run.trial074.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.2.3-705-g04cf6dd4-1jammy [3257 kB] 2026-02-17T05:32:44.130 INFO:teuthology.orchestra.run.trial074.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.2.3-705-g04cf6dd4-1jammy [979 kB] 2026-02-17T05:32:44.136 INFO:teuthology.orchestra.run.trial074.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-17T05:32:44.166 INFO:teuthology.orchestra.run.trial074.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.2.3-705-g04cf6dd4-1jammy [357 kB] 2026-02-17T05:32:44.207 INFO:teuthology.orchestra.run.trial074.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.2.3-705-g04cf6dd4-1jammy [32.9 kB] 2026-02-17T05:32:44.224 INFO:teuthology.orchestra.run.trial074.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.2.3-705-g04cf6dd4-1jammy [184 kB] 2026-02-17T05:32:44.265 INFO:teuthology.orchestra.run.trial074.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.2.3-705-g04cf6dd4-1jammy [74.1 kB] 2026-02-17T05:32:44.280 INFO:teuthology.orchestra.run.trial074.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.2.3-705-g04cf6dd4-1jammy [337 kB] 2026-02-17T05:32:44.331 INFO:teuthology.orchestra.run.trial074.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.2.3-705-g04cf6dd4-1jammy [6935 kB] 2026-02-17T05:32:44.473 INFO:teuthology.orchestra.run.trial074.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-17T05:32:44.483 INFO:teuthology.orchestra.run.trial074.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.2.3-705-g04cf6dd4-1jammy [112 kB] 2026-02-17T05:32:44.484 INFO:teuthology.orchestra.run.trial074.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.2.3-705-g04cf6dd4-1jammy [470 kB] 2026-02-17T05:32:44.485 INFO:teuthology.orchestra.run.trial074.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-17T05:32:44.527 INFO:teuthology.orchestra.run.trial074.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.2.3-705-g04cf6dd4-1jammy [26.5 MB] 2026-02-17T05:32:44.553 INFO:teuthology.orchestra.run.trial074.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-17T05:32:44.754 INFO:teuthology.orchestra.run.trial074.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-17T05:32:44.764 INFO:teuthology.orchestra.run.trial074.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-17T05:32:44.790 INFO:teuthology.orchestra.run.trial074.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-17T05:32:44.799 INFO:teuthology.orchestra.run.trial074.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-17T05:32:44.801 INFO:teuthology.orchestra.run.trial074.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-17T05:32:44.801 INFO:teuthology.orchestra.run.trial074.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-17T05:32:44.802 INFO:teuthology.orchestra.run.trial074.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-17T05:32:44.819 INFO:teuthology.orchestra.run.trial074.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2026-02-17T05:32:44.823 INFO:teuthology.orchestra.run.trial074.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2026-02-17T05:32:44.826 INFO:teuthology.orchestra.run.trial074.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2026-02-17T05:32:44.895 INFO:teuthology.orchestra.run.trial074.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5034 B] 2026-02-17T05:32:44.895 INFO:teuthology.orchestra.run.trial074.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2026-02-17T05:32:44.897 INFO:teuthology.orchestra.run.trial074.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2026-02-17T05:32:44.899 INFO:teuthology.orchestra.run.trial074.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2026-02-17T05:32:44.900 INFO:teuthology.orchestra.run.trial074.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-17T05:32:44.901 INFO:teuthology.orchestra.run.trial074.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-17T05:32:44.901 INFO:teuthology.orchestra.run.trial074.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-17T05:32:44.902 INFO:teuthology.orchestra.run.trial074.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-17T05:32:44.972 INFO:teuthology.orchestra.run.trial074.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-17T05:32:44.972 INFO:teuthology.orchestra.run.trial074.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-17T05:32:44.972 INFO:teuthology.orchestra.run.trial074.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-17T05:32:44.972 INFO:teuthology.orchestra.run.trial074.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-17T05:32:44.973 INFO:teuthology.orchestra.run.trial074.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-17T05:32:45.039 INFO:teuthology.orchestra.run.trial074.stdout:Get:40 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.2.3-705-g04cf6dd4-1jammy [5178 kB] 2026-02-17T05:32:45.044 INFO:teuthology.orchestra.run.trial074.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-17T05:32:45.046 INFO:teuthology.orchestra.run.trial074.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-17T05:32:45.046 INFO:teuthology.orchestra.run.trial074.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2026-02-17T05:32:45.047 INFO:teuthology.orchestra.run.trial074.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2026-02-17T05:32:45.047 INFO:teuthology.orchestra.run.trial074.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2026-02-17T05:32:45.116 INFO:teuthology.orchestra.run.trial074.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7320 B] 2026-02-17T05:32:45.116 INFO:teuthology.orchestra.run.trial074.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-17T05:32:45.117 INFO:teuthology.orchestra.run.trial074.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-waitress all 1.4.4-1.1ubuntu1.1 [47.0 kB] 2026-02-17T05:32:45.118 INFO:teuthology.orchestra.run.trial074.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2026-02-17T05:32:45.118 INFO:teuthology.orchestra.run.trial074.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2026-02-17T05:32:45.188 INFO:teuthology.orchestra.run.trial074.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4892 B] 2026-02-17T05:32:45.188 INFO:teuthology.orchestra.run.trial074.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2026-02-17T05:32:45.189 INFO:teuthology.orchestra.run.trial074.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2026-02-17T05:32:45.189 INFO:teuthology.orchestra.run.trial074.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2026-02-17T05:32:45.190 INFO:teuthology.orchestra.run.trial074.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.3 [181 kB] 2026-02-17T05:32:45.260 INFO:teuthology.orchestra.run.trial074.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-17T05:32:45.261 INFO:teuthology.orchestra.run.trial074.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-17T05:32:45.263 INFO:teuthology.orchestra.run.trial074.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-17T05:32:45.263 INFO:teuthology.orchestra.run.trial074.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-17T05:32:45.264 INFO:teuthology.orchestra.run.trial074.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-17T05:32:45.308 INFO:teuthology.orchestra.run.trial074.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.2.3-705-g04cf6dd4-1jammy [248 kB] 2026-02-17T05:32:45.332 INFO:teuthology.orchestra.run.trial074.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-17T05:32:45.333 INFO:teuthology.orchestra.run.trial074.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.2.3-705-g04cf6dd4-1jammy [125 kB] 2026-02-17T05:32:45.379 INFO:teuthology.orchestra.run.trial074.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.2.3-705-g04cf6dd4-1jammy [1081 kB] 2026-02-17T05:32:45.446 INFO:teuthology.orchestra.run.trial074.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.2.3-705-g04cf6dd4-1jammy [6239 kB] 2026-02-17T05:32:45.674 INFO:teuthology.orchestra.run.trial074.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-17T05:32:45.828 INFO:teuthology.orchestra.run.trial074.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-17T05:32:45.842 INFO:teuthology.orchestra.run.trial074.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.2.3-705-g04cf6dd4-1jammy [23.0 MB] 2026-02-17T05:32:46.246 INFO:teuthology.orchestra.run.trial074.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-17T05:32:46.246 INFO:teuthology.orchestra.run.trial074.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-17T05:32:46.247 INFO:teuthology.orchestra.run.trial074.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-17T05:32:46.249 INFO:teuthology.orchestra.run.trial074.stdout:Get:72 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-17T05:32:46.250 INFO:teuthology.orchestra.run.trial074.stdout:Get:73 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-17T05:32:46.253 INFO:teuthology.orchestra.run.trial074.stdout:Get:74 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-17T05:32:46.270 INFO:teuthology.orchestra.run.trial074.stdout:Get:75 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-17T05:32:46.277 INFO:teuthology.orchestra.run.trial074.stdout:Get:76 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-17T05:32:46.282 INFO:teuthology.orchestra.run.trial074.stdout:Get:77 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-17T05:32:46.318 INFO:teuthology.orchestra.run.trial074.stdout:Get:78 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-17T05:32:46.337 INFO:teuthology.orchestra.run.trial074.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.2.3-705-g04cf6dd4-1jammy [14.2 kB] 2026-02-17T05:32:46.343 INFO:teuthology.orchestra.run.trial074.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.2.3-705-g04cf6dd4-1jammy [1173 kB] 2026-02-17T05:32:46.354 INFO:teuthology.orchestra.run.trial074.stdout:Get:81 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-17T05:32:46.359 INFO:teuthology.orchestra.run.trial074.stdout:Get:82 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2026-02-17T05:32:46.360 INFO:teuthology.orchestra.run.trial074.stdout:Get:83 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2026-02-17T05:32:46.361 INFO:teuthology.orchestra.run.trial074.stdout:Get:84 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-17T05:32:46.370 INFO:teuthology.orchestra.run.trial074.stdout:Get:85 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-17T05:32:46.371 INFO:teuthology.orchestra.run.trial074.stdout:Get:86 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-17T05:32:46.377 INFO:teuthology.orchestra.run.trial074.stdout:Get:87 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-17T05:32:46.390 INFO:teuthology.orchestra.run.trial074.stdout:Get:88 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2026-02-17T05:32:46.391 INFO:teuthology.orchestra.run.trial074.stdout:Get:89 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-17T05:32:46.407 INFO:teuthology.orchestra.run.trial074.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.2.3-705-g04cf6dd4-1jammy [2503 kB] 2026-02-17T05:32:46.449 INFO:teuthology.orchestra.run.trial074.stdout:Get:91 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-17T05:32:46.452 INFO:teuthology.orchestra.run.trial074.stdout:Get:92 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-17T05:32:46.453 INFO:teuthology.orchestra.run.trial074.stdout:Get:93 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-17T05:32:46.467 INFO:teuthology.orchestra.run.trial074.stdout:Get:94 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2026-02-17T05:32:46.467 INFO:teuthology.orchestra.run.trial074.stdout:Get:95 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-17T05:32:46.468 INFO:teuthology.orchestra.run.trial074.stdout:Get:96 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-17T05:32:46.471 INFO:teuthology.orchestra.run.trial074.stdout:Get:97 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-17T05:32:46.472 INFO:teuthology.orchestra.run.trial074.stdout:Get:98 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-17T05:32:46.667 INFO:teuthology.orchestra.run.trial074.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.2.3-705-g04cf6dd4-1jammy [798 kB] 2026-02-17T05:32:46.921 INFO:teuthology.orchestra.run.trial074.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.2.3-705-g04cf6dd4-1jammy [157 kB] 2026-02-17T05:32:46.971 INFO:teuthology.orchestra.run.trial074.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.2.3-705-g04cf6dd4-1jammy [2396 kB] 2026-02-17T05:32:47.079 INFO:teuthology.orchestra.run.trial074.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.2.3-705-g04cf6dd4-1jammy [8625 kB] 2026-02-17T05:32:47.239 INFO:teuthology.orchestra.run.trial074.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.2.3-705-g04cf6dd4-1jammy [14.3 kB] 2026-02-17T05:32:47.269 INFO:teuthology.orchestra.run.trial074.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.2.3-705-g04cf6dd4-1jammy [52.1 MB] 2026-02-17T05:32:48.411 INFO:teuthology.orchestra.run.trial074.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.2.3-705-g04cf6dd4-1jammy [135 kB] 2026-02-17T05:32:48.433 INFO:teuthology.orchestra.run.trial074.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.2.3-705-g04cf6dd4-1jammy [41.0 kB] 2026-02-17T05:32:48.471 INFO:teuthology.orchestra.run.trial074.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.2.3-705-g04cf6dd4-1jammy [13.7 MB] 2026-02-17T05:32:48.718 INFO:teuthology.orchestra.run.trial074.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-bharath19-testing-2026-02-13-0903-squid/04cf6dd4be446d06026adad18a5fbf5da1fa8c0e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.2.3-705-g04cf6dd4-1jammy [92.1 kB] 2026-02-17T05:32:49.050 INFO:teuthology.orchestra.run.trial074.stdout:Fetched 178 MB in 5s (35.9 MB/s) 2026-02-17T05:32:49.067 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-17T05:32:49.223 INFO:teuthology.orchestra.run.trial074.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167438 files and directories currently installed.) 2026-02-17T05:32:49.224 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-17T05:32:49.225 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T05:32:49.247 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-17T05:32:49.259 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-17T05:32:49.260 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T05:32:49.275 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-17T05:32:49.283 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-17T05:32:49.283 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T05:32:49.300 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-17T05:32:49.312 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T05:32:49.317 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T05:32:49.349 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-17T05:32:49.359 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T05:32:49.360 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T05:32:49.375 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-17T05:32:49.383 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T05:32:49.383 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T05:32:49.403 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-17T05:32:49.411 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-17T05:32:49.411 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T05:32:49.428 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package librados2. 2026-02-17T05:32:49.436 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../007-librados2_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:49.437 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking librados2 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.479 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libnbd0. 2026-02-17T05:32:49.487 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2026-02-17T05:32:49.487 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-17T05:32:49.502 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package librbd1. 2026-02-17T05:32:49.510 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../009-librbd1_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:49.510 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking librbd1 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.551 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libcephfs2. 2026-02-17T05:32:49.559 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../010-libcephfs2_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:49.560 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libcephfs2 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.580 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-rados. 2026-02-17T05:32:49.587 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../011-python3-rados_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:49.588 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-rados (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.604 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-17T05:32:49.613 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:49.613 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-ceph-argparse (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.627 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-cephfs. 2026-02-17T05:32:49.635 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../013-python3-cephfs_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:49.636 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-cephfs (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.651 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-17T05:32:49.659 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../014-python3-ceph-common_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:49.660 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-ceph-common (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.677 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-17T05:32:49.685 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-17T05:32:49.685 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T05:32:49.699 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-prettytable. 2026-02-17T05:32:49.708 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2026-02-17T05:32:49.708 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-17T05:32:49.723 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-rbd. 2026-02-17T05:32:49.731 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../017-python3-rbd_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:49.731 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-rbd (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:49.747 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-17T05:32:49.756 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-17T05:32:49.756 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T05:32:49.774 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libreadline-dev:amd64. 2026-02-17T05:32:49.782 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2026-02-17T05:32:49.782 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2026-02-17T05:32:49.799 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2026-02-17T05:32:49.807 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2026-02-17T05:32:49.807 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-17T05:32:49.824 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package lua5.1. 2026-02-17T05:32:49.832 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2026-02-17T05:32:49.833 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2026-02-17T05:32:49.852 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package lua-any. 2026-02-17T05:32:49.860 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2026-02-17T05:32:49.860 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking lua-any (27ubuntu1) ... 2026-02-17T05:32:49.875 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package zip. 2026-02-17T05:32:49.883 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2026-02-17T05:32:49.883 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking zip (3.0-12build2) ... 2026-02-17T05:32:49.900 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package unzip. 2026-02-17T05:32:49.908 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2026-02-17T05:32:49.909 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2026-02-17T05:32:49.928 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package luarocks. 2026-02-17T05:32:49.936 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2026-02-17T05:32:49.936 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2026-02-17T05:32:49.973 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package librgw2. 2026-02-17T05:32:49.982 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../026-librgw2_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:49.982 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking librgw2 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:50.055 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-rgw. 2026-02-17T05:32:50.063 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../027-python3-rgw_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:50.063 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-rgw (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:50.078 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-17T05:32:50.086 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../028-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-17T05:32:50.086 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T05:32:50.102 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libradosstriper1. 2026-02-17T05:32:50.110 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../029-libradosstriper1_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:50.110 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libradosstriper1 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:50.129 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-common. 2026-02-17T05:32:50.137 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../030-ceph-common_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:50.137 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-common (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:50.340 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-base. 2026-02-17T05:32:50.348 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../031-ceph-base_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:50.352 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-base (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:50.416 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-17T05:32:50.424 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-17T05:32:50.425 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-17T05:32:50.440 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-cheroot. 2026-02-17T05:32:50.448 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-17T05:32:50.448 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T05:32:50.465 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-17T05:32:50.474 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-17T05:32:50.474 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-17T05:32:50.488 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-17T05:32:50.496 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-17T05:32:50.496 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-17T05:32:50.510 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-17T05:32:50.519 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-17T05:32:50.519 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-17T05:32:50.533 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-tempora. 2026-02-17T05:32:50.541 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2026-02-17T05:32:50.541 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-17T05:32:50.556 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-portend. 2026-02-17T05:32:50.564 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2026-02-17T05:32:50.565 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-17T05:32:50.578 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-17T05:32:50.586 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-17T05:32:50.586 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-17T05:32:50.601 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-17T05:32:50.609 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-17T05:32:50.610 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-17T05:32:50.632 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-natsort. 2026-02-17T05:32:50.640 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2026-02-17T05:32:50.640 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-17T05:32:50.657 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-logutils. 2026-02-17T05:32:50.665 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2026-02-17T05:32:50.665 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-logutils (0.3.3-8) ... 2026-02-17T05:32:50.680 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-mako. 2026-02-17T05:32:50.688 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2026-02-17T05:32:50.688 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-17T05:32:50.705 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-simplegeneric. 2026-02-17T05:32:50.713 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2026-02-17T05:32:50.713 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2026-02-17T05:32:50.727 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-singledispatch. 2026-02-17T05:32:50.735 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2026-02-17T05:32:50.735 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2026-02-17T05:32:50.748 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-webob. 2026-02-17T05:32:50.756 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-17T05:32:50.756 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T05:32:50.772 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-waitress. 2026-02-17T05:32:50.780 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1.1_all.deb ... 2026-02-17T05:32:50.782 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-17T05:32:50.798 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-tempita. 2026-02-17T05:32:50.806 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2026-02-17T05:32:50.807 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2026-02-17T05:32:50.820 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-paste. 2026-02-17T05:32:50.828 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2026-02-17T05:32:50.829 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2026-02-17T05:32:50.856 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2026-02-17T05:32:50.865 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2026-02-17T05:32:50.865 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2026-02-17T05:32:50.880 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-pastedeploy. 2026-02-17T05:32:50.887 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2026-02-17T05:32:50.888 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2026-02-17T05:32:50.904 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-webtest. 2026-02-17T05:32:50.912 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2026-02-17T05:32:50.913 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-webtest (2.0.35-1) ... 2026-02-17T05:32:50.928 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-pecan. 2026-02-17T05:32:50.936 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2026-02-17T05:32:50.936 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2026-02-17T05:32:50.961 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-werkzeug. 2026-02-17T05:32:50.969 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.3_all.deb ... 2026-02-17T05:32:50.969 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-17T05:32:50.989 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-17T05:32:50.997 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:50.997 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mgr-modules-core (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.026 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-17T05:32:51.034 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.034 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libsqlite3-mod-ceph (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.049 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mgr. 2026-02-17T05:32:51.057 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../057-ceph-mgr_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.057 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mgr (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.079 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mon. 2026-02-17T05:32:51.087 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../058-ceph-mon_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.088 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mon (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.143 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-17T05:32:51.151 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../059-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-17T05:32:51.152 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T05:32:51.167 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-osd. 2026-02-17T05:32:51.175 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../060-ceph-osd_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.176 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-osd (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.329 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph. 2026-02-17T05:32:51.337 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../061-ceph_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.337 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.352 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-fuse. 2026-02-17T05:32:51.360 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../062-ceph-fuse_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.360 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-fuse (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.383 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mds. 2026-02-17T05:32:51.391 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../063-ceph-mds_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.392 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mds (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.432 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package cephadm. 2026-02-17T05:32:51.440 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../064-cephadm_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:51.440 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking cephadm (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.456 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-17T05:32:51.463 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T05:32:51.464 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T05:32:51.485 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-17T05:32:51.493 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:51.494 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mgr-cephadm (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.513 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-17T05:32:51.521 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../067-python3-repoze.lru_0.7-2_all.deb ... 2026-02-17T05:32:51.522 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-17T05:32:51.536 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-routes. 2026-02-17T05:32:51.544 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../068-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-17T05:32:51.545 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T05:32:51.564 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-17T05:32:51.572 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../069-ceph-mgr-dashboard_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:51.572 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mgr-dashboard (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:51.779 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-17T05:32:51.788 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-17T05:32:51.789 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T05:32:51.827 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-joblib. 2026-02-17T05:32:51.835 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../071-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-17T05:32:51.836 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T05:32:51.862 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-17T05:32:51.870 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../072-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-17T05:32:51.871 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-17T05:32:51.885 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-sklearn. 2026-02-17T05:32:51.894 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../073-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-17T05:32:51.894 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T05:32:51.974 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-17T05:32:51.983 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../074-ceph-mgr-diskprediction-local_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:51.983 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mgr-diskprediction-local (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:52.147 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-cachetools. 2026-02-17T05:32:52.156 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../075-python3-cachetools_5.0.0-1_all.deb ... 2026-02-17T05:32:52.156 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-17T05:32:52.170 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-rsa. 2026-02-17T05:32:52.178 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../076-python3-rsa_4.8-1_all.deb ... 2026-02-17T05:32:52.179 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-17T05:32:52.195 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-google-auth. 2026-02-17T05:32:52.203 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-3_all.deb ... 2026-02-17T05:32:52.204 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-17T05:32:52.221 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-17T05:32:52.229 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-17T05:32:52.229 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T05:32:52.244 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-websocket. 2026-02-17T05:32:52.252 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../079-python3-websocket_1.2.3-1_all.deb ... 2026-02-17T05:32:52.253 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-17T05:32:52.270 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-17T05:32:52.278 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../080-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-17T05:32:52.278 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T05:32:52.373 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-17T05:32:52.382 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:52.382 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-mgr-k8sevents (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:52.397 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-17T05:32:52.405 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../082-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-17T05:32:52.412 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T05:32:52.428 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-17T05:32:52.436 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../083-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T05:32:52.436 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T05:32:52.451 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package jq. 2026-02-17T05:32:52.459 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../084-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T05:32:52.460 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-17T05:32:52.474 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package socat. 2026-02-17T05:32:52.482 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../085-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-17T05:32:52.483 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-17T05:32:52.502 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package xmlstarlet. 2026-02-17T05:32:52.510 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-17T05:32:52.510 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-17T05:32:52.546 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-test. 2026-02-17T05:32:52.555 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../087-ceph-test_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:52.555 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-test (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:53.076 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package ceph-volume. 2026-02-17T05:32:53.085 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../088-ceph-volume_19.2.3-705-g04cf6dd4-1jammy_all.deb ... 2026-02-17T05:32:53.085 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking ceph-volume (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:53.109 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-17T05:32:53.118 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../089-libcephfs-dev_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:53.118 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking libcephfs-dev (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:53.134 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package lua-socket:amd64. 2026-02-17T05:32:53.142 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../090-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2026-02-17T05:32:53.142 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-17T05:32:53.163 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package lua-sec:amd64. 2026-02-17T05:32:53.171 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../091-lua-sec_1.0.2-1_amd64.deb ... 2026-02-17T05:32:53.172 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2026-02-17T05:32:53.189 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package nvme-cli. 2026-02-17T05:32:53.197 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../092-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-17T05:32:53.197 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T05:32:53.228 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package pkg-config. 2026-02-17T05:32:53.236 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../093-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-17T05:32:53.237 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-17T05:32:53.253 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-17T05:32:53.261 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T05:32:53.261 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T05:32:53.291 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-17T05:32:53.299 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-17T05:32:53.300 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-17T05:32:53.315 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-pastescript. 2026-02-17T05:32:53.322 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2026-02-17T05:32:53.323 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2026-02-17T05:32:53.340 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-pluggy. 2026-02-17T05:32:53.349 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-17T05:32:53.349 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-17T05:32:53.366 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-psutil. 2026-02-17T05:32:53.374 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-17T05:32:53.375 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-17T05:32:53.393 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-py. 2026-02-17T05:32:53.401 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2026-02-17T05:32:53.402 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-17T05:32:53.422 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-pygments. 2026-02-17T05:32:53.430 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-17T05:32:53.430 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T05:32:53.472 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-pyinotify. 2026-02-17T05:32:53.480 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2026-02-17T05:32:53.481 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2026-02-17T05:32:53.496 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-toml. 2026-02-17T05:32:53.505 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2026-02-17T05:32:53.505 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-17T05:32:53.520 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-pytest. 2026-02-17T05:32:53.528 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-17T05:32:53.529 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T05:32:53.551 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package python3-simplejson. 2026-02-17T05:32:53.559 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-17T05:32:53.560 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-17T05:32:53.578 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-17T05:32:53.586 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-17T05:32:53.586 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-17T05:32:53.653 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package radosgw. 2026-02-17T05:32:53.662 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../106-radosgw_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:53.662 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking radosgw (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:53.773 INFO:teuthology.orchestra.run.trial074.stdout:Selecting previously unselected package rbd-fuse. 2026-02-17T05:32:53.781 INFO:teuthology.orchestra.run.trial074.stdout:Preparing to unpack .../107-rbd-fuse_19.2.3-705-g04cf6dd4-1jammy_amd64.deb ... 2026-02-17T05:32:53.781 INFO:teuthology.orchestra.run.trial074.stdout:Unpacking rbd-fuse (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:53.811 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-17T05:32:53.878 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T05:32:53.878 INFO:teuthology.orchestra.run.trial074.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T05:32:53.950 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T05:32:54.229 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-17T05:32:54.566 INFO:teuthology.orchestra.run.trial074.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-17T05:32:54.576 INFO:teuthology.orchestra.run.trial074.stdout:Setting up cephadm (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:54.630 INFO:teuthology.orchestra.run.trial074.stdout:Adding system user cephadm....done 2026-02-17T05:32:54.642 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-17T05:32:54.712 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-17T05:32:54.780 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T05:32:54.781 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-17T05:32:54.854 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-17T05:32:54.928 INFO:teuthology.orchestra.run.trial074.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T05:32:54.929 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-17T05:32:55.021 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T05:32:55.145 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-17T05:32:55.221 INFO:teuthology.orchestra.run.trial074.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2026-02-17T05:32:55.231 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2026-02-17T05:32:55.314 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-17T05:32:55.388 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-ceph-argparse (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:55.456 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T05:32:55.457 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-17T05:32:55.458 INFO:teuthology.orchestra.run.trial074.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-17T05:32:55.459 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2026-02-17T05:32:55.460 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T05:32:55.461 INFO:teuthology.orchestra.run.trial074.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2026-02-17T05:32:55.477 INFO:teuthology.orchestra.run.trial074.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2026-02-17T05:32:55.480 INFO:teuthology.orchestra.run.trial074.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2026-02-17T05:32:55.481 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T05:32:55.482 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-17T05:32:55.606 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-17T05:32:55.684 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T05:32:55.761 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-17T05:32:55.851 INFO:teuthology.orchestra.run.trial074.stdout:Setting up zip (3.0-12build2) ... 2026-02-17T05:32:55.852 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T05:32:56.101 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2026-02-17T05:32:56.178 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2026-02-17T05:32:56.179 INFO:teuthology.orchestra.run.trial074.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-17T05:32:56.180 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T05:32:56.277 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T05:32:56.400 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2026-02-17T05:32:56.530 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T05:32:56.611 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-17T05:32:56.719 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-17T05:32:56.791 INFO:teuthology.orchestra.run.trial074.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-17T05:32:56.792 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-ceph-common (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:56.876 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T05:32:57.368 INFO:teuthology.orchestra.run.trial074.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-17T05:32:57.390 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T05:32:57.395 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-17T05:32:57.470 INFO:teuthology.orchestra.run.trial074.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T05:32:57.471 INFO:teuthology.orchestra.run.trial074.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-17T05:32:57.472 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-17T05:32:57.547 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-17T05:32:57.618 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T05:32:57.619 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-17T05:32:57.699 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2026-02-17T05:32:57.771 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-logutils (0.3.3-8) ... 2026-02-17T05:32:57.847 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-17T05:32:57.921 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2026-02-17T05:32:57.994 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-17T05:32:58.071 INFO:teuthology.orchestra.run.trial074.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T05:32:58.072 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-17T05:32:58.153 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T05:32:58.154 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T05:32:58.231 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-17T05:32:58.309 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T05:32:58.394 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-17T05:32:58.468 INFO:teuthology.orchestra.run.trial074.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-17T05:32:58.469 INFO:teuthology.orchestra.run.trial074.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2026-02-17T05:32:58.470 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T05:32:58.471 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T05:32:58.606 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2026-02-17T05:32:58.685 INFO:teuthology.orchestra.run.trial074.stdout:Setting up lua-any (27ubuntu1) ... 2026-02-17T05:32:58.686 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-17T05:32:58.758 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T05:32:58.759 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-17T05:32:58.842 INFO:teuthology.orchestra.run.trial074.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-17T05:32:58.843 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-webtest (2.0.35-1) ... 2026-02-17T05:32:58.925 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-17T05:32:59.064 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-pastescript (2.0.2-4) ... 2026-02-17T05:32:59.152 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2026-02-17T05:32:59.267 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T05:32:59.268 INFO:teuthology.orchestra.run.trial074.stdout:Setting up librados2 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:59.269 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libsqlite3-mod-ceph (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:59.270 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T05:32:59.787 INFO:teuthology.orchestra.run.trial074.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2026-02-17T05:32:59.797 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libcephfs2 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:59.798 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libradosstriper1 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:59.799 INFO:teuthology.orchestra.run.trial074.stdout:Setting up librbd1 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:59.799 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mgr-modules-core (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:59.800 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-fuse (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:32:59.873 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T05:32:59.873 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T05:33:00.233 INFO:teuthology.orchestra.run.trial074.stdout:Setting up libcephfs-dev (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.234 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-rados (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.235 INFO:teuthology.orchestra.run.trial074.stdout:Setting up librgw2 (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.236 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-rbd (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.237 INFO:teuthology.orchestra.run.trial074.stdout:Setting up rbd-fuse (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.238 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-rgw (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.239 INFO:teuthology.orchestra.run.trial074.stdout:Setting up python3-cephfs (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.239 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-common (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:00.267 INFO:teuthology.orchestra.run.trial074.stdout:Adding group ceph....done 2026-02-17T05:33:00.304 INFO:teuthology.orchestra.run.trial074.stdout:Adding system user ceph....done 2026-02-17T05:33:00.314 INFO:teuthology.orchestra.run.trial074.stdout:Setting system user ceph properties....done 2026-02-17T05:33:00.321 INFO:teuthology.orchestra.run.trial074.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-17T05:33:00.403 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-17T05:33:00.678 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-17T05:33:01.042 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-test (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:01.043 INFO:teuthology.orchestra.run.trial074.stdout:Setting up radosgw (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:01.286 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:33:01.286 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T05:33:01.629 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-base (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:01.736 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-17T05:33:02.081 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mds (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:02.141 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T05:33:02.141 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T05:33:02.488 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mgr (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:02.564 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T05:33:02.565 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T05:33:02.907 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-osd (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:02.995 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T05:33:02.995 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T05:33:03.355 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mgr-k8sevents (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:03.356 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mgr-diskprediction-local (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:03.367 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mon (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:03.436 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T05:33:03.436 INFO:teuthology.orchestra.run.trial074.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T05:33:03.782 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mgr-cephadm (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:03.792 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:03.793 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-mgr-dashboard (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:03.803 INFO:teuthology.orchestra.run.trial074.stdout:Setting up ceph-volume (19.2.3-705-g04cf6dd4-1jammy) ... 2026-02-17T05:33:03.912 INFO:teuthology.orchestra.run.trial074.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2026-02-17T05:33:03.921 INFO:teuthology.orchestra.run.trial074.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.11) ... 2026-02-17T05:33:03.939 INFO:teuthology.orchestra.run.trial074.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-17T05:33:04.027 INFO:teuthology.orchestra.run.trial074.stdout:Processing triggers for install-info (6.8-4build1) ... 2026-02-17T05:33:04.513 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout:Running kernel seems to be up-to-date. 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout:The processor microcode seems to be up-to-date. 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout:No services need to be restarted. 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout:No containers need to be restarted. 2026-02-17T05:33:04.514 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:33:04.515 INFO:teuthology.orchestra.run.trial074.stdout:No user sessions are running outdated binaries. 2026-02-17T05:33:04.515 INFO:teuthology.orchestra.run.trial074.stdout: 2026-02-17T05:33:04.515 INFO:teuthology.orchestra.run.trial074.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T05:33:05.180 INFO:teuthology.orchestra.run.trial074.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-17T05:33:05.187 DEBUG:teuthology.parallel:result is None 2026-02-17T05:33:05.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:33:05.374 DEBUG:teuthology.orchestra.run.trial074:> dpkg-query -W -f '${Version}' ceph 2026-02-17T05:33:05.383 INFO:teuthology.orchestra.run.trial074.stdout:19.2.3-705-g04cf6dd4-1jammy 2026-02-17T05:33:05.383 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-705-g04cf6dd4-1jammy 2026-02-17T05:33:05.383 INFO:teuthology.task.install:The correct ceph version 19.2.3-705-g04cf6dd4-1jammy is installed. 2026-02-17T05:33:05.385 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T05:33:05.385 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:33:05.385 DEBUG:teuthology.orchestra.run.trial074:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T05:33:05.434 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T05:33:05.434 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:33:05.435 DEBUG:teuthology.orchestra.run.trial074:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T05:33:05.486 DEBUG:teuthology.orchestra.run.trial074:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T05:33:05.537 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T05:33:05.538 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:33:05.538 DEBUG:teuthology.orchestra.run.trial074:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T05:33:05.585 DEBUG:teuthology.orchestra.run.trial074:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T05:33:05.633 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T05:33:05.634 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:33:05.634 DEBUG:teuthology.orchestra.run.trial074:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T05:33:05.687 DEBUG:teuthology.orchestra.run.trial074:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T05:33:05.739 INFO:teuthology.run_tasks:Running task workunit... 2026-02-17T05:33:05.759 INFO:tasks.workunit:Pulling workunits from ref 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:33:05.759 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-17T05:33:05.759 INFO:tasks.workunit:timeout=3h 2026-02-17T05:33:05.759 INFO:tasks.workunit:cleanup=True 2026-02-17T05:33:05.760 DEBUG:teuthology.orchestra.run.trial074:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-17T05:33:05.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T05:33:05.784 INFO:teuthology.orchestra.run.trial074.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-17T05:33:05.784 DEBUG:teuthology.orchestra.run.trial074:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T05:33:05.829 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-17T05:33:05.830 DEBUG:teuthology.orchestra.run.trial074:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-17T05:33:05.873 DEBUG:teuthology.orchestra.run.trial074:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 04cf6dd4be446d06026adad18a5fbf5da1fa8c0e 2026-02-17T05:33:05.924 INFO:tasks.workunit.client.0.trial074.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-17T05:33:27.664 INFO:tasks.workunit.client.0.trial074.stderr:Note: switching to '04cf6dd4be446d06026adad18a5fbf5da1fa8c0e'. 2026-02-17T05:33:27.664 INFO:tasks.workunit.client.0.trial074.stderr: 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr:state without impacting any branches by switching back to a branch. 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr: 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr: 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr: git switch -c 2026-02-17T05:33:27.665 INFO:tasks.workunit.client.0.trial074.stderr: 2026-02-17T05:33:27.666 INFO:tasks.workunit.client.0.trial074.stderr:Or undo this operation with: 2026-02-17T05:33:27.666 INFO:tasks.workunit.client.0.trial074.stderr: 2026-02-17T05:33:27.666 INFO:tasks.workunit.client.0.trial074.stderr: git switch - 2026-02-17T05:33:27.666 INFO:tasks.workunit.client.0.trial074.stderr: 2026-02-17T05:33:27.666 INFO:tasks.workunit.client.0.trial074.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-17T05:33:27.666 INFO:tasks.workunit.client.0.trial074.stderr: 2026-02-17T05:33:27.666 INFO:tasks.workunit.client.0.trial074.stderr:HEAD is now at 04cf6dd4be4 Merge branch 'wip-squid-update-spdk' of https://github.com/tchaikov/ceph into wip-bharath19-testing-2026-02-13-0903-squid 2026-02-17T05:33:27.675 DEBUG:teuthology.orchestra.run.trial074:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-17T05:33:27.724 DEBUG:teuthology.orchestra.run.trial074:> set -ex 2026-02-17T05:33:27.724 DEBUG:teuthology.orchestra.run.trial074:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-17T05:33:27.771 INFO:tasks.workunit:Running workunits matching scrub on client.0... 2026-02-17T05:33:27.771 INFO:tasks.workunit:Running workunit scrub/osd-mapper.sh... 2026-02-17T05:33:27.772 DEBUG:teuthology.orchestra.run.trial074:workunit test scrub/osd-mapper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh 2026-02-17T05:33:27.824 INFO:tasks.workunit.client.0.trial074.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T05:33:27.829 INFO:tasks.workunit.client.0.trial074.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-mapper 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:11: run: local dir=td/osd-mapper 2026-02-17T05:33:27.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:12: run: shift 2026-02-17T05:33:27.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: export CEPH_MON=127.0.0.1:7144 2026-02-17T05:33:27.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: CEPH_MON=127.0.0.1:7144 2026-02-17T05:33:27.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:15: run: export CEPH_ARGS 2026-02-17T05:33:27.831 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: uuidgen 2026-02-17T05:33:27.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: CEPH_ARGS+='--fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none ' 2026-02-17T05:33:27.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:17: run: CEPH_ARGS+='--mon-host=127.0.0.1:7144 ' 2026-02-17T05:33:27.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:19: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T05:33:27.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: set 2026-02-17T05:33:27.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: local funcs=TEST_truncated_sna_record 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:21: run: for func in $funcs 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:22: run: setup td/osd-mapper 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-mapper 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-mapper 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:33:27.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2026-02-17T05:33:27.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:33:27.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:33:27.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:33:27.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:33:27.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:33:27.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:33:27.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:33:27.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:33:27.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:33:27.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:33:27.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:33:27.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:33:27.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:33:27.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:33:27.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:33:27.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:33:27.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:33:27.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:33:27.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2026-02-17T05:33:27.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:33:27.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:33:27.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:33:27.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11832 2026-02-17T05:33:27.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:33:27.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:33:27.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-mapper 2026-02-17T05:33:27.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:33:27.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:33:27.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:33:27.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.11832 2026-02-17T05:33:27.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:33:27.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-mapper 1' TERM HUP INT 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:23: run: TEST_truncated_sna_record td/osd-mapper 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:45: TEST_truncated_sna_record: local dir=td/osd-mapper 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='4' ['pool_name']='test') 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: local -A cluster_conf 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:52: TEST_truncated_sna_record: local extr_dbg=3 2026-02-17T05:33:27.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: (( extr_dbg > 1 )) 2026-02-17T05:33:27.852 INFO:tasks.workunit.client.0.trial074.stdout:Dir: td/osd-mapper 2026-02-17T05:33:27.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: echo 'Dir: td/osd-mapper' 2026-02-17T05:33:27.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:54: TEST_truncated_sna_record: standard_scrub_cluster td/osd-mapper cluster_conf 2026-02-17T05:33:27.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-mapper 2026-02-17T05:33:27.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-17T05:33:27.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-17T05:33:27.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-17T05:33:27.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:234: standard_scrub_cluster: local poolname=test 2026-02-17T05:33:27.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: args['pool_name']=test 2026-02-17T05:33:27.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: local extra_pars= 2026-02-17T05:33:27.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:237: standard_scrub_cluster: local debug_msg=dbg 2026-02-17T05:33:27.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:240: standard_scrub_cluster: local saved_echo_flag=x 2026-02-17T05:33:27.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: set +x 2026-02-17T05:33:28.149 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 b079fce9-58d5-4c09-803b-7964cb440b9c 2026-02-17T05:33:28.252 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:33:28.277 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:28.271+0000 7f3f7764d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:28.278 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:28.275+0000 7f3f7764d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:28.279 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:28.275+0000 7f3f7764d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:28.280 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:28.275+0000 7f3f7764d8c0 -1 bdev(0x55f4903e1c00 td/osd-mapper/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:33:28.280 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:28.275+0000 7f3f7764d8c0 -1 bluestore(td/osd-mapper/0) _read_fsid unparsable uuid 2026-02-17T05:33:31.139 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:33:31.335 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:33:31.359 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:31.355+0000 7f66f11868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:31.361 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:31.355+0000 7f66f11868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:31.362 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:31.355+0000 7f66f11868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:31.485 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:33:32.637 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:33:32.876 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:32.871+0000 7f66f11868c0 -1 Falling back to public interface 2026-02-17T05:33:33.788 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:33:34.936 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:33:35.888 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:35.883+0000 7f66f11868c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:33:36.089 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:33:37.233 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:33:37.383 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/592570301,v1:127.0.0.1:6803/592570301] [v2:127.0.0.1:6804/592570301,v1:127.0.0.1:6805/592570301] exists,up b079fce9-58d5-4c09-803b-7964cb440b9c 2026-02-17T05:33:37.388 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 c2feca2a-91c2-441d-b210-821f89463c40 2026-02-17T05:33:37.551 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:33:37.576 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:37.571+0000 7f01869f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:37.578 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:37.571+0000 7f01869f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:37.579 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:37.575+0000 7f01869f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:37.580 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:37.575+0000 7f01869f68c0 -1 bdev(0x563887b0dc00 td/osd-mapper/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:33:37.580 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:37.575+0000 7f01869f68c0 -1 bluestore(td/osd-mapper/1) _read_fsid unparsable uuid 2026-02-17T05:33:39.900 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:33:40.087 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:33:40.112 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:40.107+0000 7f242720b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:40.113 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:40.107+0000 7f242720b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:40.114 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:40.107+0000 7f242720b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:40.240 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:33:41.387 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:33:41.616 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:41.611+0000 7f242720b8c0 -1 Falling back to public interface 2026-02-17T05:33:42.534 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:33:43.681 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:33:44.564 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:44.559+0000 7f242720b8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:33:44.830 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:33:45.981 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:33:46.127 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4165674397,v1:127.0.0.1:6811/4165674397] [v2:127.0.0.1:6812/4165674397,v1:127.0.0.1:6813/4165674397] exists,up c2feca2a-91c2-441d-b210-821f89463c40 2026-02-17T05:33:46.132 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 e207faae-22c4-4ad7-9c7c-abd87a717732 2026-02-17T05:33:46.295 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:33:46.321 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:46.315+0000 7ffaee8cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:46.322 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:46.319+0000 7ffaee8cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:46.324 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:46.319+0000 7ffaee8cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:46.324 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:46.319+0000 7ffaee8cc8c0 -1 bdev(0x56509ba13c00 td/osd-mapper/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:33:46.324 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:46.319+0000 7ffaee8cc8c0 -1 bluestore(td/osd-mapper/2) _read_fsid unparsable uuid 2026-02-17T05:33:48.621 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:33:48.816 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:33:48.841 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:48.835+0000 7f3772da48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:48.842 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:48.835+0000 7f3772da48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:48.843 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:48.839+0000 7f3772da48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:33:48.960 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:33:50.114 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:33:50.368 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:50.363+0000 7f3772da48c0 -1 Falling back to public interface 2026-02-17T05:33:51.261 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:33:52.413 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:33:53.338 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:33:53.330+0000 7f3772da48c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:33:53.565 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:33:54.711 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:33:54.864 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3900473518,v1:127.0.0.1:6819/3900473518] [v2:127.0.0.1:6820/3900473518,v1:127.0.0.1:6821/3900473518] exists,up e207faae-22c4-4ad7-9c7c-abd87a717732 2026-02-17T05:33:55.061 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T05:33:56.545 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:33:57.856 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705668 2026-02-17T05:33:58.010 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574850 2026-02-17T05:33:59.832 INFO:tasks.workunit.client.0.trial074.stdout:standard_scrub_cluster: dbg: test pool is test 1 2026-02-17T05:33:59.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:55: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_not_scrubbing 1 2026-02-17T05:33:59.893 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T05:33:59.893 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2026-02-17T05:33:59.893 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:33:59.899 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T05:33:59.900 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2026-02-17T05:33:59.900 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:33:59.906 INFO:tasks.workunit.client.0.trial074.stdout:osd.2: { 2026-02-17T05:33:59.906 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2026-02-17T05:33:59.906 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:33:59.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:56: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_scrubbing 1 2026-02-17T05:33:59.987 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T05:33:59.987 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2026-02-17T05:33:59.987 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:33:59.993 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T05:33:59.993 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2026-02-17T05:33:59.993 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:33:59.999 INFO:tasks.workunit.client.0.trial074.stdout:osd.2: { 2026-02-17T05:33:59.999 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2026-02-17T05:34:00.000 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:00.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:58: TEST_truncated_sna_record: local osdn=3 2026-02-17T05:34:00.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:59: TEST_truncated_sna_record: local poolid=1 2026-02-17T05:34:00.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:60: TEST_truncated_sna_record: local poolname=test 2026-02-17T05:34:00.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:61: TEST_truncated_sna_record: local objname=objxxx 2026-02-17T05:34:00.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:64: TEST_truncated_sna_record: make_a_clone test objxxx snap01 snap02 2026-02-17T05:34:00.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-17T05:34:00.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-17T05:34:00.076 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap01 2026-02-17T05:34:00.178 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap02 2026-02-17T05:34:00.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:65: TEST_truncated_sna_record: make_a_clone test objxxx snap13 2026-02-17T05:34:00.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-17T05:34:00.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-17T05:34:00.280 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap13 2026-02-17T05:34:00.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:66: TEST_truncated_sna_record: make_a_clone test objxxx snap24 snap25 2026-02-17T05:34:00.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-17T05:34:00.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-17T05:34:00.384 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap24 2026-02-17T05:34:00.485 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap25 2026-02-17T05:34:00.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: echo 24252 2026-02-17T05:34:00.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: rados -p test put objxxx - 2026-02-17T05:34:00.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-17T05:34:00.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: jq -r .pgid 2026-02-17T05:34:00.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: local pgid=1.3 2026-02-17T05:34:00.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-17T05:34:00.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: jq -r '.up[0]' 2026-02-17T05:34:00.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: local osd=1 2026-02-17T05:34:00.810 INFO:tasks.workunit.client.0.trial074.stdout:pgid is 1.3 (primary: osd.1) 2026-02-17T05:34:00.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:72: TEST_truncated_sna_record: echo 'pgid is 1.3 (primary: osd.1)' 2026-02-17T05:34:00.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:74: TEST_truncated_sna_record: set_query_debug 1.3 2026-02-17T05:34:00.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:303: set_query_debug: local pgid=1.3 2026-02-17T05:34:00.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: ceph pg dump pgs_brief 2026-02-17T05:34:00.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: awk -v 'pg=^1.3' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-17T05:34:00.949 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs_brief 2026-02-17T05:34:00.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: local prim_osd=1 2026-02-17T05:34:00.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:307: set_query_debug: echo 'Setting scrub debug data. Primary for 1.3 is 1' 2026-02-17T05:34:00.959 INFO:tasks.workunit.client.0.trial074.stdout:Setting scrub debug data. Primary for 1.3 is 1 2026-02-17T05:34:00.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: get_asok_path osd.1 2026-02-17T05:34:00.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T05:34:00.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T05:34:00.961 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:34:00.961 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:34:00.961 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:34:00.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.11832/ceph-osd.1.asok 2026-02-17T05:34:00.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: CEPH_ARGS= 2026-02-17T05:34:00.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.11832/ceph-osd.1.asok scrubdebug 1.3 set sessions 2026-02-17T05:34:01.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:01.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2026-02-17T05:34:01.050 INFO:tasks.workunit.client.0.trial074.stdout:{"success":true}{ 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: "name": "objxxx", 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 5, 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: "id": 2, 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [ 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: "id": 1, 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap01" 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:01.051 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "id": 2, 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap02" 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6, 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "overlaps": [] 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "id": 3, 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [ 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "id": 3, 2026-02-17T05:34:01.052 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap13" 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6, 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: "overlaps": [] 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: "id": 5, 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [ 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: "id": 4, 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap24" 2026-02-17T05:34:01.053 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: "id": 5, 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap25" 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6, 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: "overlaps": [] 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: "id": "head", 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [], 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6 2026-02-17T05:34:01.054 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:01.055 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T05:34:01.055 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:01.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:80: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-17T05:34:01.117 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:34:01.117 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T05:34:01.117 INFO:tasks.workunit.client.0.trial074.stdout: "must": true, 2026-02-17T05:34:01.117 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "0.000000" 2026-02-17T05:34:01.117 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:01.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:84: TEST_truncated_sna_record: sleep 3 2026-02-17T05:34:04.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:85: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T05:34:04.267 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:34:04.268 INFO:tasks.workunit.client.0.trial074.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2026-02-17T05:33:55.168306+0000 22'7 22:33 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.268 INFO:tasks.workunit.client.0.trial074.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:33:55.168442+0000 0'0 22:29 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.268 INFO:tasks.workunit.client.0.trial074.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:33:55.869423+0000 0'0 17:14 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.268 INFO:tasks.workunit.client.0.trial074.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:33:55.168305+0000 0'0 22:29 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.269 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:04.269 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:34:04.269 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:34:04.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:86: TEST_truncated_sna_record: grep -a -q -- 'event: --^^^^---- ScrubFinished' td/osd-mapper/osd.1.log 2026-02-17T05:34:04.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:90: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T05:34:04.415 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:34:04.415 INFO:tasks.workunit.client.0.trial074.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2026-02-17T05:33:55.168306+0000 22'7 22:33 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.415 INFO:tasks.workunit.client.0.trial074.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:33:55.168442+0000 0'0 22:29 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.415 INFO:tasks.workunit.client.0.trial074.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:33:55.869423+0000 0'0 17:14 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.415 INFO:tasks.workunit.client.0.trial074.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:33:55.168305+0000 0'0 22:29 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:04.415 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:04.415 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:34:04.416 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:34:04.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:91: TEST_truncated_sna_record: ceph osd set noscrub 2026-02-17T05:34:04.631 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T05:34:04.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:92: TEST_truncated_sna_record: ceph osd set nodeep-scrub 2026-02-17T05:34:04.836 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T05:34:04.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:93: TEST_truncated_sna_record: sleep 5 2026-02-17T05:34:09.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:94: TEST_truncated_sna_record: grep -a -q -v ERR td/osd-mapper/osd.1.log 2026-02-17T05:34:09.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:97: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2026-02-17T05:34:09.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:34:09.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:34:09.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:34:09.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:34:09.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:34:11.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:34:11.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:11.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-17T05:34:11.176 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.170+0000 7efc8b503bc0 1 bdev(0x5647eb8f1800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:11.176 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.170+0000 7efc8b503bc0 1 bdev(0x5647eb8f1800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:11.176 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.170+0000 7efc8b503bc0 1 bdev(0x5647eb8f1800 td/osd-mapper/0/block) close 2026-02-17T05:34:11.456 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bdev(0x5647eb8f1800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:11.456 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bdev(0x5647eb8f1800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:11.457 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T05:34:11.457 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bdev(0x5647eb8f1c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:11.457 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bdev(0x5647eb8f1c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:11.457 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T05:34:11.457 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bluefs mount 2026-02-17T05:34:11.457 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:11.459 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.460 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.460 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.450+0000 7efc8b503bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.460 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.454+0000 7efc8b503bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.460 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.454+0000 7efc8b503bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.460 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.454+0000 7efc8b503bc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T05:34:11.460 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.454+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:11.472 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:11.472 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2051 2026-02-17T05:34:11.472 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2026-02-17T05:34:11.473 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T05:34:11.473 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2026-02-17T05:34:11.473 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 freelist init 2026-02-17T05:34:11.473 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 freelist _read_cfg 2026-02-17T05:34:11.474 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 20 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 7.24794e-07 2026-02-17T05:34:11.474 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bluefs umount 2026-02-17T05:34:11.475 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.466+0000 7efc8b503bc0 1 bdev(0x5647eb8f1c00 td/osd-mapper/0/block) close 2026-02-17T05:34:11.744 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bdev(0x5647eb8f1c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:11.744 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bdev(0x5647eb8f1c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:11.744 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T05:34:11.744 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs mount 2026-02-17T05:34:11.744 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:11.745 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.745 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.745 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.745 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.745 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:11.746 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluefs mount shared_bdev_used = 7929856 2026-02-17T05:34:11.746 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.738+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:11.758 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.750+0000 7efc8b503bc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:11.759 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T05:34:11.759 INFO:tasks.workunit.client.0.trial074.stdout:00000000 0b 05 58 02 00 00 31 35 e4 66 e9 e3 40 70 a5 4d |..X...15.f..@p.M| 2026-02-17T05:34:11.759 INFO:tasks.workunit.client.0.trial074.stdout:00000010 83 66 f3 74 93 f8 00 00 00 00 18 00 00 00 00 00 |.f.t............| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:* 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T05:34:11.760 INFO:tasks.workunit.client.0.trial074.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T05:34:11.761 INFO:tasks.workunit.client.0.trial074.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T05:34:11.762 INFO:tasks.workunit.client.0.trial074.stdout:00000220 72 65 18 00 00 00 05 00 00 00 b0 79 fc e9 58 d5 |re.........y..X.| 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:00000230 4c 09 80 3b 79 64 cb 44 0b 9c 00 00 00 00 00 00 |L..;yd.D........| 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:00000240 00 00 18 00 00 00 b0 fd 93 69 c5 35 6e 37 01 00 |.........i.5n7..| 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:0000025e 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:11.763 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:0000003f 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.764 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:11.765 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:11.766 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.767 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T05:34:11.768 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:11.769 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.750+0000 7efc8b503bc0 1 bluefs umount 2026-02-17T05:34:11.769 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:11.750+0000 7efc8b503bc0 1 bdev(0x5647eb8f1c00 td/osd-mapper/0/block) close 2026-02-17T05:34:12.023 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.018+0000 7efc8b503bc0 1 freelist shutdown 2026-02-17T05:34:12.024 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.018+0000 7efc8b503bc0 1 bdev(0x5647eb8f1800 td/osd-mapper/0/block) close 2026-02-17T05:34:12.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:12.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T05:34:12.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: grep -a SNA_ 2026-02-17T05:34:12.254 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.246+0000 7f0c880cbbc0 1 bdev(0x561f15ec5800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:12.254 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.246+0000 7f0c880cbbc0 1 bdev(0x561f15ec5800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:12.255 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.246+0000 7f0c880cbbc0 1 bdev(0x561f15ec5800 td/osd-mapper/2/block) close 2026-02-17T05:34:12.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bdev(0x561f15ec5800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:12.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bdev(0x561f15ec5800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:12.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T05:34:12.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bdev(0x561f15ec5c00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:12.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bdev(0x561f15ec5c00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:12.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T05:34:12.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs mount 2026-02-17T05:34:12.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:12.539 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.539 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.539 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.539 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.539 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.540 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T05:34:12.540 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.530+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:12.558 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.550+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:12.558 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.550+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2050 2026-02-17T05:34:12.558 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.550+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2026-02-17T05:34:12.558 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.550+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T05:34:12.558 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.550+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2026-02-17T05:34:12.559 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.550+0000 7f0c880cbbc0 1 freelist init 2026-02-17T05:34:12.559 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.550+0000 7f0c880cbbc0 1 freelist _read_cfg 2026-02-17T05:34:12.562 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.554+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 11 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 3.81471e-07 2026-02-17T05:34:12.562 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.554+0000 7f0c880cbbc0 1 bluefs umount 2026-02-17T05:34:12.562 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.554+0000 7f0c880cbbc0 1 bdev(0x561f15ec5c00 td/osd-mapper/2/block) close 2026-02-17T05:34:12.815 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bdev(0x561f15ec5c00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:12.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bdev(0x561f15ec5c00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:12.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T05:34:12.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs mount 2026-02-17T05:34:12.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:12.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:12.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T05:34:12.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.810+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:12.830 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.822+0000 7f0c880cbbc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:12.830 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.822+0000 7f0c880cbbc0 1 bluefs umount 2026-02-17T05:34:12.830 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:12.822+0000 7f0c880cbbc0 1 bdev(0x561f15ec5c00 td/osd-mapper/2/block) close 2026-02-17T05:34:13.103 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:13.098+0000 7f0c880cbbc0 1 freelist shutdown 2026-02-17T05:34:13.104 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:13.098+0000 7f0c880cbbc0 1 bdev(0x561f15ec5800 td/osd-mapper/2/block) close 2026-02-17T05:34:13.323 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:13.323 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:13.323 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:13.323 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:13.323 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:13.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:13.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: grep -a SNA_ /tmp/oo2.dump 2026-02-17T05:34:13.324 INFO:tasks.workunit.client.0.trial074.stderr:grep: /tmp/oo2.dump: No such file or directory 2026-02-17T05:34:13.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:13.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T05:34:13.920 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T05:34:13.920 INFO:tasks.workunit.client.0.trial074.stdout:00000000 0b 05 58 02 00 00 31 35 e4 66 e9 e3 40 70 a5 4d |..X...15.f..@p.M| 2026-02-17T05:34:13.920 INFO:tasks.workunit.client.0.trial074.stdout:00000010 83 66 f3 74 93 f8 02 00 00 00 18 00 00 00 00 00 |.f.t............| 2026-02-17T05:34:13.920 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T05:34:13.920 INFO:tasks.workunit.client.0.trial074.stdout:* 2026-02-17T05:34:13.920 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T05:34:13.921 INFO:tasks.workunit.client.0.trial074.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T05:34:13.922 INFO:tasks.workunit.client.0.trial074.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:00000220 72 65 18 00 00 00 0d 00 00 00 e2 07 fa ae 22 c4 |re............".| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:00000230 4a d7 9c 7c ab d8 7a 71 77 32 00 00 00 00 00 00 |J..|..zqw2......| 2026-02-17T05:34:13.923 INFO:tasks.workunit.client.0.trial074.stdout:00000240 00 00 18 00 00 00 c2 fd 93 69 af 56 84 12 01 00 |.........i.V....| 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout:0000025e 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:13.924 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T05:34:13.925 INFO:tasks.workunit.client.0.trial074.stdout:0000003f 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:13.926 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:13.927 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.928 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:13.929 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:13.930 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.930 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T05:34:13.930 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:13.930 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T05:34:13.930 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:14.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T05:34:14.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2026-02-17T05:34:14.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T05:34:14.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: wc -l 2026-02-17T05:34:15.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: local num_sna_b4=5 2026-02-17T05:34:15.498 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: expr 3 - 1 2026-02-17T05:34:15.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: seq 0 2 2026-02-17T05:34:15.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:15.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/0 2026-02-17T05:34:15.500 INFO:tasks.workunit.client.0.trial074.stdout:corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0) 2026-02-17T05:34:15.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0)' 2026-02-17T05:34:15.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-17T05:34:15.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-17T05:34:15.519 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.514+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:15.520 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.514+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:15.520 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.514+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199800 td/osd-mapper/0/block) close 2026-02-17T05:34:15.807 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:15.807 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:15.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T05:34:15.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:15.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:15.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T05:34:15.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bluefs mount 2026-02-17T05:34:15.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.802+0000 7f5d79b0ebc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:15.812 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.806+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:15.812 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.806+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:15.813 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.806+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:15.813 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.806+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:15.813 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.806+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:15.813 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.806+0000 7f5d79b0ebc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T05:34:15.813 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.806+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:15.834 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.826+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:15.834 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.826+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2051 2026-02-17T05:34:15.834 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.826+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2026-02-17T05:34:15.834 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.826+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T05:34:15.834 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.826+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2026-02-17T05:34:15.835 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.826+0000 7f5d79b0ebc0 1 freelist init 2026-02-17T05:34:15.835 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.826+0000 7f5d79b0ebc0 1 freelist _read_cfg 2026-02-17T05:34:15.838 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.830+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 20 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 7.24794e-07 2026-02-17T05:34:15.838 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.830+0000 7f5d79b0ebc0 1 bluefs umount 2026-02-17T05:34:15.838 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:15.830+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199c00 td/osd-mapper/0/block) close 2026-02-17T05:34:16.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199c00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T05:34:16.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199c00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:16.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T05:34:16.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs mount 2026-02-17T05:34:16.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:16.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:16.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:16.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:16.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:16.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:16.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluefs mount shared_bdev_used = 7929856 2026-02-17T05:34:16.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.090+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:16.116 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.110+0000 7f5d79b0ebc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:00000000 0b 05 58 02 00 00 31 35 e4 66 e9 e3 40 70 a5 4d |..X...15.f..@p.M| 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:00000010 83 66 f3 74 93 f8 00 00 00 00 18 00 00 00 00 00 |.f.t............| 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:* 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T05:34:16.118 INFO:tasks.workunit.client.0.trial074.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T05:34:16.119 INFO:tasks.workunit.client.0.trial074.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T05:34:16.120 INFO:tasks.workunit.client.0.trial074.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:00000220 72 65 18 00 00 00 05 00 00 00 b0 79 fc e9 58 d5 |re.........y..X.| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:00000230 4c 09 80 3b 79 64 cb 44 0b 9c 00 00 00 00 00 00 |L..;yd.D........| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:00000240 00 00 18 00 00 00 b0 fd 93 69 c5 35 6e 37 01 00 |.........i.5n7..| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout:0000025e 2026-02-17T05:34:16.121 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:16.122 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:0000003f 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:16.123 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:16.124 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:16.125 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.126 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T05:34:16.127 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:16.128 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.110+0000 7f5d79b0ebc0 1 bluefs umount 2026-02-17T05:34:16.128 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.110+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199c00 td/osd-mapper/0/block) close 2026-02-17T05:34:16.375 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.370+0000 7f5d79b0ebc0 1 freelist shutdown 2026-02-17T05:34:16.376 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:16.370+0000 7f5d79b0ebc0 1 bdev(0x55a9f0199800 td/osd-mapper/0/block) close 2026-02-17T05:34:16.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-17T05:34:16.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-17T05:34:16.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T05:34:17.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:17.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:17.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-17T05:34:17.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-17T05:34:17.678 INFO:tasks.workunit.client.0.trial074.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:17.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-17T05:34:17.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.ynCMP1pH88_the_val 2026-02-17T05:34:17.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:17.680 INFO:tasks.workunit.client.0.trial074.stdout:Value dumped in: /tmp/tmp.ynCMP1pH88_the_val 2026-02-17T05:34:17.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.ynCMP1pH88_the_val' 2026-02-17T05:34:17.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.ynCMP1pH88_the_val 2026-02-17T05:34:18.270 INFO:tasks.workunit.client.0.trial074.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-17T05:34:18.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:18.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.ynCMP1pH88_the_val 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stdout:0000073 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T05:34:18.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:19.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.ynCMP1pH88_the_val 2026-02-17T05:34:20.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:20.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:20.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2026-02-17T05:34:20.953 INFO:tasks.workunit.client.0.trial074.stdout:corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1) 2026-02-17T05:34:20.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1)' 2026-02-17T05:34:20.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-17T05:34:20.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T05:34:20.965 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:20.958+0000 7f7cd1a75bc0 1 bdev(0x561bb5039800 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T05:34:20.965 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:20.958+0000 7f7cd1a75bc0 1 bdev(0x561bb5039800 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:20.966 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:20.958+0000 7f7cd1a75bc0 1 bdev(0x561bb5039800 td/osd-mapper/1/block) close 2026-02-17T05:34:21.256 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bdev(0x561bb5039800 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T05:34:21.256 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bdev(0x561bb5039800 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:21.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T05:34:21.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bdev(0x561bb5039c00 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T05:34:21.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bdev(0x561bb5039c00 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:21.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2026-02-17T05:34:21.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs mount 2026-02-17T05:34:21.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:21.259 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.259 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.260 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.260 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.260 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.260 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T05:34:21.260 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.250+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:21.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.266+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:21.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.266+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _open_super_meta old nid_max 2051 2026-02-17T05:34:21.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.266+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _open_super_meta old blobid_max 20480 2026-02-17T05:34:21.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.266+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T05:34:21.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.266+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _open_super_meta min_alloc_size 0x1000 2026-02-17T05:34:21.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.266+0000 7f7cd1a75bc0 1 freelist init 2026-02-17T05:34:21.276 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.266+0000 7f7cd1a75bc0 1 freelist _read_cfg 2026-02-17T05:34:21.276 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.270+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _init_alloc loaded 100 GiB in 17 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 6.10353e-07 2026-02-17T05:34:21.277 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.270+0000 7f7cd1a75bc0 1 bluefs umount 2026-02-17T05:34:21.277 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.270+0000 7f7cd1a75bc0 1 bdev(0x561bb5039c00 td/osd-mapper/1/block) close 2026-02-17T05:34:21.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bdev(0x561bb5039c00 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T05:34:21.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bdev(0x561bb5039c00 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:21.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2026-02-17T05:34:21.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs mount 2026-02-17T05:34:21.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:21.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.538 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.538 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.538 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:21.538 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T05:34:21.538 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.530+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:21.550 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.542+0000 7f7cd1a75bc0 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:21.550 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000000 0b 05 58 02 00 00 31 35 e4 66 e9 e3 40 70 a5 4d |..X...15.f..@p.M| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000010 83 66 f3 74 93 f8 01 00 00 00 18 00 00 00 00 00 |.f.t............| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:* 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T05:34:21.551 INFO:tasks.workunit.client.0.trial074.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T05:34:21.552 INFO:tasks.workunit.client.0.trial074.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T05:34:21.553 INFO:tasks.workunit.client.0.trial074.stdout:00000220 72 65 18 00 00 00 09 00 00 00 c2 fe ca 2a 91 c2 |re...........*..| 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:00000230 44 1d b2 10 82 1f 89 46 3c 40 00 00 00 00 00 00 |D......F<@......| 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:00000240 00 00 18 00 00 00 b9 fd 93 69 2d 5b 71 20 01 00 |.........i-[q ..| 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:0000025e 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:21.554 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:0000003f 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.555 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:21.556 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:21.557 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:21.558 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T05:34:21.559 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:21.560 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.542+0000 7f7cd1a75bc0 1 bluefs umount 2026-02-17T05:34:21.560 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.542+0000 7f7cd1a75bc0 1 bdev(0x561bb5039c00 td/osd-mapper/1/block) close 2026-02-17T05:34:21.824 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.818+0000 7f7cd1a75bc0 1 freelist shutdown 2026-02-17T05:34:21.824 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:21.818+0000 7f7cd1a75bc0 1 bdev(0x561bb5039800 td/osd-mapper/1/block) close 2026-02-17T05:34:22.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T05:34:22.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-17T05:34:22.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T05:34:23.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:23.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:23.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-17T05:34:23.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-17T05:34:23.131 INFO:tasks.workunit.client.0.trial074.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:23.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-17T05:34:23.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.a5WtKmLGfE_the_val 2026-02-17T05:34:23.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:23.133 INFO:tasks.workunit.client.0.trial074.stdout:Value dumped in: /tmp/tmp.a5WtKmLGfE_the_val 2026-02-17T05:34:23.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.a5WtKmLGfE_the_val' 2026-02-17T05:34:23.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.a5WtKmLGfE_the_val 2026-02-17T05:34:23.722 INFO:tasks.workunit.client.0.trial074.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-17T05:34:24.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:24.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.a5WtKmLGfE_the_val 2026-02-17T05:34:24.213 INFO:tasks.workunit.client.0.trial074.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-17T05:34:24.213 INFO:tasks.workunit.client.0.trial074.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-17T05:34:24.213 INFO:tasks.workunit.client.0.trial074.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-17T05:34:24.214 INFO:tasks.workunit.client.0.trial074.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-17T05:34:24.214 INFO:tasks.workunit.client.0.trial074.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-17T05:34:24.214 INFO:tasks.workunit.client.0.trial074.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-17T05:34:24.214 INFO:tasks.workunit.client.0.trial074.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-17T05:34:24.214 INFO:tasks.workunit.client.0.trial074.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-17T05:34:24.214 INFO:tasks.workunit.client.0.trial074.stdout:0000073 2026-02-17T05:34:24.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T05:34:24.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:25.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.a5WtKmLGfE_the_val 2026-02-17T05:34:26.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:26.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:26.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/2 2026-02-17T05:34:26.409 INFO:tasks.workunit.client.0.trial074.stdout:corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2) 2026-02-17T05:34:26.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2)' 2026-02-17T05:34:26.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-17T05:34:26.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T05:34:26.420 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.413+0000 7fa52c5acbc0 1 bdev(0x5558a7543800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:26.420 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.413+0000 7fa52c5acbc0 1 bdev(0x5558a7543800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:26.420 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.413+0000 7fa52c5acbc0 1 bdev(0x5558a7543800 td/osd-mapper/2/block) close 2026-02-17T05:34:26.699 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bdev(0x5558a7543800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:26.700 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bdev(0x5558a7543800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:26.700 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T05:34:26.700 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bdev(0x5558a7543c00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:26.700 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bdev(0x5558a7543c00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:26.700 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T05:34:26.700 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs mount 2026-02-17T05:34:26.700 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:26.703 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.703 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.703 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.703 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.704 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.704 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T05:34:26.704 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.693+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:26.715 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:26.715 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2050 2026-02-17T05:34:26.715 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2026-02-17T05:34:26.715 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T05:34:26.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2026-02-17T05:34:26.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 freelist init 2026-02-17T05:34:26.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 freelist _read_cfg 2026-02-17T05:34:26.717 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 11 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffc8000, fragmentation 3.81471e-07 2026-02-17T05:34:26.718 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bluefs umount 2026-02-17T05:34:26.718 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.709+0000 7fa52c5acbc0 1 bdev(0x5558a7543c00 td/osd-mapper/2/block) close 2026-02-17T05:34:26.980 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bdev(0x5558a7543c00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T05:34:26.980 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bdev(0x5558a7543c00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T05:34:26.980 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T05:34:26.981 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs mount 2026-02-17T05:34:26.981 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T05:34:26.981 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.981 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T05:34:26.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T05:34:26.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.973+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T05:34:26.998 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.989+0000 7fa52c5acbc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:00000000 0b 05 58 02 00 00 31 35 e4 66 e9 e3 40 70 a5 4d |..X...15.f..@p.M| 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:00000010 83 66 f3 74 93 f8 02 00 00 00 18 00 00 00 00 00 |.f.t............| 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:* 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T05:34:27.000 INFO:tasks.workunit.client.0.trial074.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T05:34:27.001 INFO:tasks.workunit.client.0.trial074.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T05:34:27.002 INFO:tasks.workunit.client.0.trial074.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:00000220 72 65 18 00 00 00 0d 00 00 00 e2 07 fa ae 22 c4 |re............".| 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:00000230 4a d7 9c 7c ab d8 7a 71 77 32 00 00 00 00 00 00 |J..|..zqw2......| 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:00000240 00 00 18 00 00 00 c2 fd 93 69 af 56 84 12 01 00 |.........i.V....| 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:00000250 00 00 01 00 00 00 01 00 00 00 18 00 00 00 |..............| 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:0000025e 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:27.003 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout:0000003f 2026-02-17T05:34:27.004 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:00000047 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:27.005 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:27.006 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:27.007 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:0000003b 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T05:34:27.008 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:27.009 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.989+0000 7fa52c5acbc0 1 bluefs umount 2026-02-17T05:34:27.009 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:26.989+0000 7fa52c5acbc0 1 bdev(0x5558a7543c00 td/osd-mapper/2/block) close 2026-02-17T05:34:27.272 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:27.265+0000 7fa52c5acbc0 1 freelist shutdown 2026-02-17T05:34:27.272 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:27.265+0000 7fa52c5acbc0 1 bdev(0x5558a7543800 td/osd-mapper/2/block) close 2026-02-17T05:34:27.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T05:34:27.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-17T05:34:27.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T05:34:28.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:28.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:28.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-17T05:34:28.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-17T05:34:28.567 INFO:tasks.workunit.client.0.trial074.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:28.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-17T05:34:28.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.MGc2TrBqt2_the_val 2026-02-17T05:34:28.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:28.569 INFO:tasks.workunit.client.0.trial074.stdout:Value dumped in: /tmp/tmp.MGc2TrBqt2_the_val 2026-02-17T05:34:28.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.MGc2TrBqt2_the_val' 2026-02-17T05:34:28.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.MGc2TrBqt2_the_val 2026-02-17T05:34:29.166 INFO:tasks.workunit.client.0.trial074.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-17T05:34:29.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:29.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.MGc2TrBqt2_the_val 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-17T05:34:29.650 INFO:tasks.workunit.client.0.trial074.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-17T05:34:29.651 INFO:tasks.workunit.client.0.trial074.stdout:0000073 2026-02-17T05:34:29.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T05:34:29.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:34:30.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.MGc2TrBqt2_the_val 2026-02-17T05:34:31.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:31.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:129: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 ' 2026-02-17T05:34:31.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: echo --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 2026-02-17T05:34:31.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T05:34:31.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:34:31.814 INFO:tasks.workunit.client.0.trial074.stdout:Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/ 2026-02-17T05:34:31.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: echo 'Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/' 2026-02-17T05:34:31.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: expr 3 - 1 2026-02-17T05:34:31.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: seq 0 2 2026-02-17T05:34:31.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:31.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T05:34:31.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 0 2026-02-17T05:34:31.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2026-02-17T05:34:31.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:34:31.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/0 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/0' 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/0/journal' 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:34:31.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11832/$cluster-$name.asok' 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11832/$cluster-$name.asok' 2026-02-17T05:34:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/0 2026-02-17T05:34:31.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:34:31.823 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:34:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/0 --osd-journal=td/osd-mapper/0/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.11832/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:34:31.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/0/whoami 2026-02-17T05:34:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:34:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:34:31.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:34:31.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:34:31.840 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:31.833+0000 7fded2a058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:31.841 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:31.833+0000 7fded2a058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:31.842 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:31.833+0000 7fded2a058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:31.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:34:31.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:34:31.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:34:31.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:34:31.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:34:31.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:31.976 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:34:31.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:34:31.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:31.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:34:32.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:33.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:33.089+0000 7fded2a058c0 -1 Falling back to public interface 2026-02-17T05:34:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:33.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:33.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:34:33.133 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:34:33.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:33.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:34:33.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:34.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:34.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:34.291 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:34:34.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:34:34.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:34.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:34:34.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:35.447 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:34:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:34:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:34:35.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:36.081 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:36.073+0000 7fded2a058c0 -1 osd.0 24 log_to_monitors true 2026-02-17T05:34:36.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:36.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:36.599 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:34:36.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:34:36.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:36.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:34:36.754 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 27 up_thru 27 down_at 25 last_clean_interval [5,24) [v2:127.0.0.1:6802/3125047602,v1:127.0.0.1:6803/3125047602] [v2:127.0.0.1:6804/3125047602,v1:127.0.0.1:6805/3125047602] exists,up b079fce9-58d5-4c09-803b-7964cb440b9c 2026-02-17T05:34:36.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:34:36.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:34:36.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:34:36.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:36.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T05:34:36.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 1 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/1 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:34:36.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/1' 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/1/journal' 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:34:36.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11832/$cluster-$name.asok' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11832/$cluster-$name.asok' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:34:36.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/1 2026-02-17T05:34:36.763 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:34:36.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:34:36.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/1 --osd-journal=td/osd-mapper/1/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.11832/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:34:36.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/1/whoami 2026-02-17T05:34:36.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:34:36.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:34:36.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:34:36.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:34:36.785 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:36.777+0000 7fe49e3038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:36.786 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:36.777+0000 7fe49e3038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:36.787 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:36.777+0000 7fe49e3038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:36.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:34:36.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:34:36.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:34:36.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:34:36.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:34:36.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:36.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:34:36.917 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:34:36.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:36.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:34:37.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:38.029 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:38.021+0000 7fe49e3038c0 -1 Falling back to public interface 2026-02-17T05:34:38.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:38.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:38.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:34:38.071 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:34:38.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:38.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:34:38.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:39.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:39.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:39.221 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:34:39.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:34:39.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:39.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:34:39.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:40.373 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:34:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:34:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:34:40.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:41.000 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:40.989+0000 7fe49e3038c0 -1 osd.1 24 log_to_monitors true 2026-02-17T05:34:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:41.531 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:34:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:34:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:34:41.689 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 30 up_thru 30 down_at 25 last_clean_interval [9,24) [v2:127.0.0.1:6810/3867304005,v1:127.0.0.1:6811/3867304005] [v2:127.0.0.1:6812/3867304005,v1:127.0.0.1:6813/3867304005] exists,up c2feca2a-91c2-441d-b210-821f89463c40 2026-02-17T05:34:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:34:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 2 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/2 2026-02-17T05:34:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/2' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/2/journal' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:34:41.696 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11832/$cluster-$name.asok' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11832/$cluster-$name.asok' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:34:41.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:34:41.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/2 2026-02-17T05:34:41.698 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:34:41.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T05:34:41.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=3135e466-e9e3-4070-a54d-8366f37493f8 --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/2 --osd-journal=td/osd-mapper/2/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.11832/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:34:41.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/2/whoami 2026-02-17T05:34:41.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T05:34:41.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:34:41.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:34:41.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:34:41.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:41.705+0000 7ff36c3f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:41.717 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:41.709+0000 7ff36c3f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:41.718 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:41.709+0000 7ff36c3f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:34:41.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T05:34:41.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:34:41.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:34:41.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:34:41.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:34:41.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:41.853 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:34:41.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:34:41.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:41.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:34:42.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:42.962 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:42.953+0000 7ff36c3f58c0 -1 Falling back to public interface 2026-02-17T05:34:43.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:43.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:43.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:34:43.011 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:34:43.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:43.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:34:43.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:34:44.169 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:34:44.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:44.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:34:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:45.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:45.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:45.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:34:45.323 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:34:45.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:45.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:34:45.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:34:45.961 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:34:45.953+0000 7ff36c3f58c0 -1 osd.2 24 log_to_monitors true 2026-02-17T05:34:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:34:46.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:34:46.479 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:34:46.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:34:46.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:34:46.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:34:46.635 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 35 up_thru 35 down_at 25 last_clean_interval [13,24) [v2:127.0.0.1:6818/3361553082,v1:127.0.0.1:6819/3361553082] [v2:127.0.0.1:6820/3361553082,v1:127.0.0.1:6821/3361553082] exists,up e207faae-22c4-4ad7-9c7c-abd87a717732 2026-02-17T05:34:46.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:34:46.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:34:46.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:34:46.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:136: TEST_truncated_sna_record: sleep 1 2026-02-17T05:34:47.638 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: expr 3 - 1 2026-02-17T05:34:47.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: seq 0 2 2026-02-17T05:34:47.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:47.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.0 version 2026-02-17T05:34:47.703 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:34:47.703 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19.2.3-705-g04cf6dd4", 2026-02-17T05:34:47.704 INFO:tasks.workunit.client.0.trial074.stdout: "release": "squid", 2026-02-17T05:34:47.704 INFO:tasks.workunit.client.0.trial074.stdout: "release_type": "stable" 2026-02-17T05:34:47.704 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:47.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:47.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.1 version 2026-02-17T05:34:47.771 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:34:47.772 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19.2.3-705-g04cf6dd4", 2026-02-17T05:34:47.772 INFO:tasks.workunit.client.0.trial074.stdout: "release": "squid", 2026-02-17T05:34:47.772 INFO:tasks.workunit.client.0.trial074.stdout: "release_type": "stable" 2026-02-17T05:34:47.772 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:47.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T05:34:47.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.2 version 2026-02-17T05:34:47.839 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:34:47.839 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19.2.3-705-g04cf6dd4", 2026-02-17T05:34:47.840 INFO:tasks.workunit.client.0.trial074.stdout: "release": "squid", 2026-02-17T05:34:47.840 INFO:tasks.workunit.client.0.trial074.stdout: "release_type": "stable" 2026-02-17T05:34:47.840 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:47.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:142: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: "name": "objxxx", 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 5, 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: "id": 2, 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [ 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: "id": 1, 2026-02-17T05:34:47.864 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap01" 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: "id": 2, 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap02" 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6, 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: "overlaps": [] 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: "id": 3, 2026-02-17T05:34:47.865 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [ 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: "id": 3, 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap13" 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6, 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: "overlaps": [] 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: "id": 5, 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [ 2026-02-17T05:34:47.866 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: "id": 4, 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap24" 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: "id": 5, 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: "name": "snap25" 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6, 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: "overlaps": [] 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T05:34:47.867 INFO:tasks.workunit.client.0.trial074.stdout: "id": "head", 2026-02-17T05:34:47.868 INFO:tasks.workunit.client.0.trial074.stdout: "snapshots": [], 2026-02-17T05:34:47.868 INFO:tasks.workunit.client.0.trial074.stdout: "size": 6 2026-02-17T05:34:47.868 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T05:34:47.868 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T05:34:47.868 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:47.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:145: TEST_truncated_sna_record: ceph osd unset nodeep-scrub 2026-02-17T05:34:48.068 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is unset 2026-02-17T05:34:48.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:146: TEST_truncated_sna_record: ceph osd unset noscrub 2026-02-17T05:34:48.273 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is unset 2026-02-17T05:34:48.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-17T05:34:48.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: jq -r '.up[0]' 2026-02-17T05:34:48.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: local cur_prim=1 2026-02-17T05:34:48.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:150: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T05:34:48.574 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:34:48.575 INFO:tasks.workunit.client.0.trial074.stdout:1.3 4 0 4 0 0 24 0 0 7 0 7 active+undersized+degraded 2026-02-17T05:34:41.258452+0000 22'7 34:54 [1,0] 1 [1,0] 1 22'7 2026-02-17T05:34:01.378595+0000 22'7 2026-02-17T05:34:01.378595+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:34:01.378595+0000 4 0 2026-02-17T05:34:48.575 INFO:tasks.workunit.client.0.trial074.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 peering 2026-02-17T05:34:46.116050+0000 0'0 34:31 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 no scrub is scheduled 0 0 2026-02-17T05:34:48.575 INFO:tasks.workunit.client.0.trial074.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 peering 2026-02-17T05:34:41.158965+0000 0'0 30:12 [0,1] 0 [0,1] 0 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 no scrub is scheduled 0 0 2026-02-17T05:34:48.575 INFO:tasks.workunit.client.0.trial074.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-17T05:34:41.257882+0000 0'0 34:21 [1,0] 1 [1,0] 1 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:48.575 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:48.575 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:34:48.575 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:34:48.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:151: TEST_truncated_sna_record: sleep 2 2026-02-17T05:34:50.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:152: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-17T05:34:50.646 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:34:50.647 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T05:34:50.647 INFO:tasks.workunit.client.0.trial074.stdout: "must": true, 2026-02-17T05:34:50.647 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "0.000000" 2026-02-17T05:34:50.647 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:50.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:153: TEST_truncated_sna_record: sleep 5 2026-02-17T05:34:55.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:154: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2026-02-17T05:34:46.219469+0000 22'7 37:68 [1,2,0] 1 [1,2,0] 1 22'7 2026-02-17T05:34:01.378595+0000 22'7 2026-02-17T05:34:01.378595+0000 0 0 queued for deep scrub 4 0 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:34:46.219657+0000 0'0 39:45 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:34:46.219811+0000 0'0 37:25 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:34:46.219605+0000 0'0 37:33 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:34:55.794 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:34:55.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:34:55.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T05:34:55.803 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:34:51.129+0000 7fe47aa7e640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2026-02-17T05:34:55.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:156: TEST_truncated_sna_record: grep -a -q ERR td/osd-mapper/osd.1.log 2026-02-17T05:34:55.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T05:34:55.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: wc -l 2026-02-17T05:34:55.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: local prev_err_cnt=1 2026-02-17T05:34:55.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:161: TEST_truncated_sna_record: echo 'prev count: 1' 2026-02-17T05:34:55.809 INFO:tasks.workunit.client.0.trial074.stdout:prev count: 1 2026-02-17T05:34:55.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:164: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-17T05:34:55.875 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:34:55.875 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T05:34:55.875 INFO:tasks.workunit.client.0.trial074.stdout: "must": true, 2026-02-17T05:34:55.875 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "0.000000" 2026-02-17T05:34:55.875 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:34:55.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:165: TEST_truncated_sna_record: sleep 5 2026-02-17T05:35:00.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:166: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T05:35:01.022 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:35:01.022 INFO:tasks.workunit.client.0.trial074.stdout:1.3 4 0 0 0 0 24 0 0 7 0 7 active+clean 2026-02-17T05:34:51.136171+0000 22'7 39:80 [1,2,0] 1 [1,2,0] 1 22'7 2026-02-17T05:34:51.136112+0000 22'7 2026-02-17T05:34:51.136112+0000 0 0 queued for deep scrub 4 0 2026-02-17T05:35:01.022 INFO:tasks.workunit.client.0.trial074.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:34:46.219657+0000 0'0 39:45 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:35:01.022 INFO:tasks.workunit.client.0.trial074.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:34:46.219811+0000 0'0 39:28 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:35:01.022 INFO:tasks.workunit.client.0.trial074.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:34:46.219605+0000 0'0 39:36 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T05:33:55.058251+0000 0'0 2026-02-17T05:33:55.058251+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:33:55.058251+0000 0 0 2026-02-17T05:35:01.022 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:35:01.022 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:35:01.023 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:01.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:35:01.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T05:35:01.032 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:34:51.129+0000 7fe47aa7e640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2026-02-17T05:35:01.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T05:35:01.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: wc -l 2026-02-17T05:35:01.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: local current_err_cnt=1 2026-02-17T05:35:01.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T05:35:01.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: echo 'current count: 1' 2026-02-17T05:35:01.036 INFO:tasks.workunit.client.0.trial074.stdout:current count: 1 2026-02-17T05:35:01.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:170: TEST_truncated_sna_record: (( current_err_cnt == prev_err_cnt )) 2026-02-17T05:35:01.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:171: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2026-02-17T05:35:01.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:35:01.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:35:01.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:35:01.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:35:01.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:35:02.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:35:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:172: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2026-02-17T05:35:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T05:35:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T05:35:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_' 2026-02-17T05:35:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:174: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T05:35:03.425 INFO:tasks.workunit.client.0.trial074.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:35:03.425 INFO:tasks.workunit.client.0.trial074.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T05:35:03.426 INFO:tasks.workunit.client.0.trial074.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T05:35:03.426 INFO:tasks.workunit.client.0.trial074.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T05:35:03.426 INFO:tasks.workunit.client.0.trial074.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:35:03.426 INFO:tasks.workunit.client.0.trial074.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T05:35:03.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T05:35:03.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2026-02-17T05:35:03.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T05:35:03.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: wc -l 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: local num_sna_full=5 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:177: TEST_truncated_sna_record: (( num_sna_full == num_sna_b4 )) 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:178: TEST_truncated_sna_record: return 0 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:24: run: teardown td/osd-mapper 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:35:04.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:35:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:35:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:35:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:35:04.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:35:04.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:35:04.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:35:04.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:35:04.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:35:04.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:35:04.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:35:04.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:35:04.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:35:04.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:35:04.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:35:04.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:35:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:35:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2026-02-17T05:35:04.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:35:04.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:04.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:35:04.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11832 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-mapper 0 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:35:04.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:35:04.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:35:04.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:35:04.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:35:04.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:35:04.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:35:04.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:35:04.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:35:04.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:35:04.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:35:04.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:35:04.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:35:04.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:35:04.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:35:04.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:35:04.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:35:04.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T05:35:04.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2026-02-17T05:35:04.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:35:04.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:04.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11832 2026-02-17T05:35:04.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11832 2026-02-17T05:35:04.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:35:04.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:35:04.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T05:35:04.655 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T05:35:04.655 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T05:35:04.704 INFO:tasks.workunit:Running workunit scrub/osd-recovery-scrub.sh... 2026-02-17T05:35:04.705 DEBUG:teuthology.orchestra.run.trial074:workunit test scrub/osd-recovery-scrub.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh 2026-02-17T05:35:04.758 INFO:tasks.workunit.client.0.trial074.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T05:35:04.761 INFO:tasks.workunit.client.0.trial074.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T05:35:04.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-recovery-scrub 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:20: run: local dir=td/osd-recovery-scrub 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:21: run: shift 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: export CEPH_MON=127.0.0.1:7124 2026-02-17T05:35:04.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: CEPH_MON=127.0.0.1:7124 2026-02-17T05:35:04.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:24: run: export CEPH_ARGS 2026-02-17T05:35:04.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: uuidgen 2026-02-17T05:35:04.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: CEPH_ARGS+='--fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none ' 2026-02-17T05:35:04.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-17T05:35:04.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:27: run: CEPH_ARGS+='--osd-op-queue=wpq ' 2026-02-17T05:35:04.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:29: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T05:35:04.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: set 2026-02-17T05:35:04.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T05:35:04.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: local funcs=TEST_recovery_scrub_1 2026-02-17T05:35:04.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:31: run: for func in $funcs 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:32: run: TEST_recovery_scrub_1 td/osd-recovery-scrub 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:39: TEST_recovery_scrub_1: local dir=td/osd-recovery-scrub 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:40: TEST_recovery_scrub_1: local poolname=test 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:42: TEST_recovery_scrub_1: TESTDATA=testdata.31194 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:43: TEST_recovery_scrub_1: OSDS=4 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:44: TEST_recovery_scrub_1: PGS=1 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:45: TEST_recovery_scrub_1: OBJECTS=100 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:46: TEST_recovery_scrub_1: ERRORS=0 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:48: TEST_recovery_scrub_1: setup td/osd-recovery-scrub 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-recovery-scrub 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-recovery-scrub 2026-02-17T05:35:04.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:35:04.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:35:04.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:35:04.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:35:04.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:35:04.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:35:04.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:35:04.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:35:04.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:35:04.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:35:04.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:35:04.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:35:04.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:35:04.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:35:04.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2026-02-17T05:35:04.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:35:04.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:04.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:04.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.31194 2026-02-17T05:35:04.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:35:04.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:35:04.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-recovery-scrub 2026-02-17T05:35:04.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:35:04.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:04.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:04.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.31194 2026-02-17T05:35:04.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:35:04.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T05:35:04.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T05:35:04.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:35:04.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-recovery-scrub 1' TERM HUP INT 2026-02-17T05:35:04.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:49: TEST_recovery_scrub_1: run_mon td/osd-recovery-scrub a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T05:35:04.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-recovery-scrub 2026-02-17T05:35:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:35:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:35:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:35:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-recovery-scrub/a 2026-02-17T05:35:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-recovery-scrub/a --run-dir=td/osd-recovery-scrub --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T05:35:04.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:35:04.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:35:04.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:35:04.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:35:04.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:04.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:04.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:04.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-recovery-scrub/a '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --mon-cluster-log-file=td/osd-recovery-scrub/log --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T05:35:04.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:35:04.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:35:04.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:35:04.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:35:04.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:35:04.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.31194/ceph-mon.a.asok 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:35:04.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.31194/ceph-mon.a.asok config get fsid 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:35:04.891 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:35:04.892 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:35:04.892 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:04.892 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:04.892 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.31194/ceph-mon.a.asok 2026-02-17T05:35:04.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:35:04.893 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.31194/ceph-mon.a.asok config get mon_host 2026-02-17T05:35:04.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:50: TEST_recovery_scrub_1: run_mgr td/osd-recovery-scrub x 2026-02-17T05:35:04.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-recovery-scrub 2026-02-17T05:35:04.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:35:04.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:35:04.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:35:04.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-recovery-scrub/x 2026-02-17T05:35:04.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:35:05.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:35:05.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:35:05.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:35:05.039 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:35:05.039 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:05.039 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:05.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:05.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:35:05.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-recovery-scrub/x '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:35:05.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:51: TEST_recovery_scrub_1: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 ' 2026-02-17T05:35:05.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:52: TEST_recovery_scrub_1: ceph_osd_args+='--osd_scrub_backoff_ratio=0 ' 2026-02-17T05:35:05.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:53: TEST_recovery_scrub_1: ceph_osd_args+='--osd_stats_update_period_not_scrubbing=3 ' 2026-02-17T05:35:05.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:54: TEST_recovery_scrub_1: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2026-02-17T05:35:05.059 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: expr 4 - 1 2026-02-17T05:35:05.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: seq 0 3 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 0 --osd_scrub_during_recovery=false 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/0 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/0' 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/0/journal' 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:35:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T05:35:05.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/0 2026-02-17T05:35:05.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:35:05.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ec30ff0f-78e3-406a-9657-bccaa3120305 2026-02-17T05:35:05.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 ec30ff0f-78e3-406a-9657-bccaa3120305' 2026-02-17T05:35:05.064 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 ec30ff0f-78e3-406a-9657-bccaa3120305 2026-02-17T05:35:05.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:35:05.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAJ/pNpS19CBBAAqcNZzAw544J2zdVVk4PYEg== 2026-02-17T05:35:05.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAJ/pNpS19CBBAAqcNZzAw544J2zdVVk4PYEg=="}' 2026-02-17T05:35:05.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ec30ff0f-78e3-406a-9657-bccaa3120305 -i td/osd-recovery-scrub/0/new.json 2026-02-17T05:35:05.162 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:35:05.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/0/new.json 2026-02-17T05:35:05.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQAJ/pNpS19CBBAAqcNZzAw544J2zdVVk4PYEg== --osd-uuid ec30ff0f-78e3-406a-9657-bccaa3120305 2026-02-17T05:35:05.186 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:05.180+0000 7f2981ac58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:05.188 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:05.180+0000 7f2981ac58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:05.189 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:05.184+0000 7f2981ac58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:05.189 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:05.184+0000 7f2981ac58c0 -1 bdev(0x5556d5a77c00 td/osd-recovery-scrub/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:35:05.189 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:05.184+0000 7f2981ac58c0 -1 bluestore(td/osd-recovery-scrub/0) _read_fsid unparsable uuid 2026-02-17T05:35:07.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/0/keyring 2026-02-17T05:35:07.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:35:07.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:35:07.956 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:35:07.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:35:08.142 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:35:08.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:35:08.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T05:35:08.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:35:08.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:35:08.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:35:08.157 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:08.152+0000 7f14867708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:08.158 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:08.152+0000 7f14867708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:08.159 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:08.152+0000 7f14867708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:08.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:35:08.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:35:08.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:35:08.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:35:08.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:35:08.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:08.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:35:08.291 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:35:08.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:08.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:35:08.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:09.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:09.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:09.444 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:35:09.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:35:09.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:09.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:35:09.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:09.650 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:09.644+0000 7f14867708c0 -1 Falling back to public interface 2026-02-17T05:35:10.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:10.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:10.598 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:35:10.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:35:10.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:10.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:35:10.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:11.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:11.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:11.748 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:35:11.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:35:11.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:11.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:35:11.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:12.648 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:12.640+0000 7f14867708c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:35:12.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:12.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:12.903 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:35:12.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:35:12.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:12.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:35:13.049 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4031889487,v1:127.0.0.1:6803/4031889487] [v2:127.0.0.1:6804/4031889487,v1:127.0.0.1:6805/4031889487] exists,up ec30ff0f-78e3-406a-9657-bccaa3120305 2026-02-17T05:35:13.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 1 --osd_scrub_during_recovery=false 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:35:13.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/1 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/1' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/1/journal' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T05:35:13.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:35:13.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/1 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c15121b1-a5f6-4e71-be4e-939c46d64931 2026-02-17T05:35:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 c15121b1-a5f6-4e71-be4e-939c46d64931' 2026-02-17T05:35:13.058 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 c15121b1-a5f6-4e71-be4e-939c46d64931 2026-02-17T05:35:13.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:35:13.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAR/pNpv6aaAxAAI3aDzws97ds1W8aEhvqtHw== 2026-02-17T05:35:13.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAR/pNpv6aaAxAAI3aDzws97ds1W8aEhvqtHw=="}' 2026-02-17T05:35:13.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c15121b1-a5f6-4e71-be4e-939c46d64931 -i td/osd-recovery-scrub/1/new.json 2026-02-17T05:35:13.210 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:35:13.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/1/new.json 2026-02-17T05:35:13.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQAR/pNpv6aaAxAAI3aDzws97ds1W8aEhvqtHw== --osd-uuid c15121b1-a5f6-4e71-be4e-939c46d64931 2026-02-17T05:35:13.245 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:13.240+0000 7f9999f0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:13.246 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:13.240+0000 7f9999f0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:13.247 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:13.240+0000 7f9999f0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:13.248 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:13.240+0000 7f9999f0b8c0 -1 bdev(0x55f0bf72dc00 td/osd-recovery-scrub/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:35:13.248 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:13.240+0000 7f9999f0b8c0 -1 bluestore(td/osd-recovery-scrub/1) _read_fsid unparsable uuid 2026-02-17T05:35:15.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/1/keyring 2026-02-17T05:35:15.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:35:15.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:35:15.552 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:35:15.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:35:15.743 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:35:15.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:35:15.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T05:35:15.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:35:15.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:35:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:35:15.767 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:15.760+0000 7f4961a6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:15.768 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:15.760+0000 7f4961a6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:15.769 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:15.764+0000 7f4961a6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:35:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:35:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:35:15.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:35:15.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:35:15.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:15.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:35:15.892 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:35:15.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:15.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:35:16.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:17.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:17.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:17.046 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:35:17.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:35:17.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:17.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:35:17.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:17.282 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:17.276+0000 7f4961a6a8c0 -1 Falling back to public interface 2026-02-17T05:35:18.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:18.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:18.195 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:35:18.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:35:18.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:18.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:35:18.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:19.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:19.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:19.346 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:35:19.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:35:19.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:19.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:35:19.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:20.252 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:20.244+0000 7f4961a6a8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:35:20.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:20.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:20.497 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:35:20.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:35:20.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:20.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:35:20.646 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2404116822,v1:127.0.0.1:6811/2404116822] [v2:127.0.0.1:6812/2404116822,v1:127.0.0.1:6813/2404116822] exists,up c15121b1-a5f6-4e71-be4e-939c46d64931 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 2 --osd_scrub_during_recovery=false 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/2 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T05:35:20.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/2' 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/2/journal' 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:35:20.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:35:20.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:35:20.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:35:20.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:35:20.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T05:35:20.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/2 2026-02-17T05:35:20.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:35:20.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e12399f6-8e6d-4e0c-a8b8-a9038f2c7d56 2026-02-17T05:35:20.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 e12399f6-8e6d-4e0c-a8b8-a9038f2c7d56' 2026-02-17T05:35:20.651 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 e12399f6-8e6d-4e0c-a8b8-a9038f2c7d56 2026-02-17T05:35:20.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:35:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAY/pNp4nc2JxAA0RnmCYdvGln1cWZo6pnQtg== 2026-02-17T05:35:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAY/pNp4nc2JxAA0RnmCYdvGln1cWZo6pnQtg=="}' 2026-02-17T05:35:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e12399f6-8e6d-4e0c-a8b8-a9038f2c7d56 -i td/osd-recovery-scrub/2/new.json 2026-02-17T05:35:20.816 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:35:20.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/2/new.json 2026-02-17T05:35:20.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQAY/pNp4nc2JxAA0RnmCYdvGln1cWZo6pnQtg== --osd-uuid e12399f6-8e6d-4e0c-a8b8-a9038f2c7d56 2026-02-17T05:35:20.842 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:20.836+0000 7f61562e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:20.843 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:20.836+0000 7f61562e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:20.845 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:20.840+0000 7f61562e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:20.845 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:20.840+0000 7f61562e48c0 -1 bdev(0x5637e025fc00 td/osd-recovery-scrub/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:35:20.845 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:20.840+0000 7f61562e48c0 -1 bluestore(td/osd-recovery-scrub/2) _read_fsid unparsable uuid 2026-02-17T05:35:23.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/2/keyring 2026-02-17T05:35:23.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:35:23.118 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:35:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:35:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:35:23.310 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:35:23.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:35:23.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T05:35:23.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:35:23.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:35:23.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:35:23.334 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:23.328+0000 7f1f4f7248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:23.335 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:23.328+0000 7f1f4f7248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:23.336 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:23.332+0000 7f1f4f7248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:23.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:35:23.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:35:23.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:35:23.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:35:23.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:35:23.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:23.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:35:23.458 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:35:23.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:23.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:35:23.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:24.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:24.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:24.608 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:35:24.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:35:24.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:24.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:35:24.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:24.830 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:24.824+0000 7f1f4f7248c0 -1 Falling back to public interface 2026-02-17T05:35:25.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:25.760 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:35:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:35:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:35:25.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:26.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:26.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:26.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:35:26.912 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:35:26.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:26.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:35:27.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:27.801 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:27.791+0000 7f1f4f7248c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:35:28.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:28.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:28.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:35:28.066 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:35:28.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:28.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:35:28.214 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1680851399,v1:127.0.0.1:6819/1680851399] [v2:127.0.0.1:6820/1680851399,v1:127.0.0.1:6821/1680851399] exists,up e12399f6-8e6d-4e0c-a8b8-a9038f2c7d56 2026-02-17T05:35:28.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:35:28.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 3 --osd_scrub_during_recovery=false 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/3 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:35:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/3' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/3/journal' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:35:28.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T05:35:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/3 2026-02-17T05:35:28.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:35:28.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=652909b1-6656-4c2b-b661-ac3e0c1a259e 2026-02-17T05:35:28.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 652909b1-6656-4c2b-b661-ac3e0c1a259e' 2026-02-17T05:35:28.222 INFO:tasks.workunit.client.0.trial074.stdout:add osd3 652909b1-6656-4c2b-b661-ac3e0c1a259e 2026-02-17T05:35:28.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:35:28.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAg/pNpbzd2DRAATemjKbDYpwOgJd0vUnx3Rw== 2026-02-17T05:35:28.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAg/pNpbzd2DRAATemjKbDYpwOgJd0vUnx3Rw=="}' 2026-02-17T05:35:28.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 652909b1-6656-4c2b-b661-ac3e0c1a259e -i td/osd-recovery-scrub/3/new.json 2026-02-17T05:35:28.387 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:35:28.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/3/new.json 2026-02-17T05:35:28.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQAg/pNpbzd2DRAATemjKbDYpwOgJd0vUnx3Rw== --osd-uuid 652909b1-6656-4c2b-b661-ac3e0c1a259e 2026-02-17T05:35:28.413 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:28.407+0000 7feb82df28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:28.415 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:28.407+0000 7feb82df28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:28.416 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:28.407+0000 7feb82df28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:28.417 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:28.407+0000 7feb82df28c0 -1 bdev(0x557814035c00 td/osd-recovery-scrub/3/block) open stat got: (1) Operation not permitted 2026-02-17T05:35:28.417 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:28.407+0000 7feb82df28c0 -1 bluestore(td/osd-recovery-scrub/3) _read_fsid unparsable uuid 2026-02-17T05:35:30.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/3/keyring 2026-02-17T05:35:30.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:35:30.719 INFO:tasks.workunit.client.0.trial074.stdout:adding osd3 key to auth repository 2026-02-17T05:35:30.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T05:35:30.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:35:30.912 INFO:tasks.workunit.client.0.trial074.stdout:start osd.3 2026-02-17T05:35:30.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T05:35:30.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=5cefd430-101f-44b9-91f1-2005769cd3ac --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31194/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T05:35:30.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:35:30.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:35:30.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:35:30.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:30.927+0000 7f178af008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:30.937 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:30.927+0000 7f178af008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:30.938 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:30.931+0000 7f178af008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:31.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:35:31.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:32.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:32.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:32.215 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:35:32.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:35:32.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:32.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:35:32.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:32.431 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:32.423+0000 7f178af008c0 -1 Falling back to public interface 2026-02-17T05:35:33.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:33.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:33.366 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:35:33.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:35:33.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:33.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:35:33.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:34.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:34.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:34.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:35:34.517 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:35:34.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:34.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:35:34.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:35:35.432 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:35:35.423+0000 7f178af008c0 -1 osd.3 0 log_to_monitors true 2026-02-17T05:35:35.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:35:35.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:35:35.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:35:35.669 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:35:35.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:35:35.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:35:35.815 INFO:tasks.workunit.client.0.trial074.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3796185772,v1:127.0.0.1:6827/3796185772] [v2:127.0.0.1:6828/3796185772,v1:127.0.0.1:6829/3796185772] exists,up 652909b1-6656-4c2b-b661-ac3e0c1a259e 2026-02-17T05:35:35.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:35:35.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:35:35.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:35:35.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:61: TEST_recovery_scrub_1: create_pool test 1 1 2026-02-17T05:35:35.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T05:35:35.993 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T05:35:36.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:35:37.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:62: TEST_recovery_scrub_1: wait_for_clean 2026-02-17T05:35:37.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:35:37.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:35:37.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:35:37.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:35:37.005 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:35:37.005 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:35:37.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:35:37.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:35:37.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:35:37.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:35:37.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:35:37.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:35:37.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:35:37.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:35:37.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:35:37.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:35:37.253 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:35:37.253 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:35:37.253 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:35:37.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:35:37.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:35:37.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:35:37.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T05:35:37.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T05:35:37.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T05:35:37.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:35:37.327 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:35:37.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T05:35:37.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T05:35:37.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705669' 2026-02-17T05:35:37.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:35:37.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:35:37.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-17T05:35:37.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-17T05:35:37.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705669 2-55834574851' 2026-02-17T05:35:37.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:35:37.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:35:37.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444034 2026-02-17T05:35:37.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444034 2026-02-17T05:35:37.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705669 2-55834574851 3-73014444034' 2026-02-17T05:35:37.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:35:37.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T05:35:37.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:35:37.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:35:37.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T05:35:37.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:35:37.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T05:35:37.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T05:35:37.546 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836486 2026-02-17T05:35:37.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:35:37.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T05:35:37.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:35:38.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:35:38.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:35:38.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T05:35:38.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:35:39.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:35:39.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:35:39.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836486 2026-02-17T05:35:39.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:35:39.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T05:35:39.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:35:39.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:35:39.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T05:35:40.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:35:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T05:35:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T05:35:40.001 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705669 2026-02-17T05:35:40.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:35:40.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T05:35:40.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:35:40.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-17T05:35:40.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:35:40.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:35:40.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-17T05:35:40.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:35:40.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-17T05:35:40.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-17T05:35:40.165 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574851 2026-02-17T05:35:40.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:35:40.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574851 2026-02-17T05:35:40.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:35:40.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-73014444034 2026-02-17T05:35:40.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:35:40.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:35:40.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-73014444034 2026-02-17T05:35:40.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:35:40.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444034 2026-02-17T05:35:40.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 73014444034' 2026-02-17T05:35:40.320 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 73014444034 2026-02-17T05:35:40.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:35:40.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444034 -lt 73014444034 2026-02-17T05:35:40.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:35:40.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:35:40.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:35:40.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:35:40.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:35:40.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:35:40.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:35:40.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:35:40.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:35:40.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:35:40.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:35:40.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:35:40.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:35:40.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:35:40.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:35:40.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:35:40.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:35:40.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:35:40.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: ceph osd dump 2026-02-17T05:35:40.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: grep '^pool.*['\'']test['\'']' 2026-02-17T05:35:40.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: awk '{ print $2 }' 2026-02-17T05:35:41.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: poolid=1 2026-02-17T05:35:41.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:65: TEST_recovery_scrub_1: ceph pg dump pgs 2026-02-17T05:35:41.292 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:35:41.292 INFO:tasks.workunit.client.0.trial074.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T05:35:36.386133+0000 0'0 20:11 [1] 1 [1] 1 0'0 2026-02-17T05:35:35.986597+0000 0'0 2026-02-17T05:35:35.986597+0000 0 0 periodic scrub scheduled @ 2026-02-18T10:29:58.856683+0000 0 0 2026-02-17T05:35:41.292 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:35:41.292 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:35:41.293 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:41.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:67: TEST_recovery_scrub_1: dd if=/dev/urandom of=testdata.31194 bs=1M count=50 2026-02-17T05:35:41.439 INFO:tasks.workunit.client.0.trial074.stderr:50+0 records in 2026-02-17T05:35:41.439 INFO:tasks.workunit.client.0.trial074.stderr:50+0 records out 2026-02-17T05:35:41.439 INFO:tasks.workunit.client.0.trial074.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.137779 s, 381 MB/s 2026-02-17T05:35:41.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: seq 1 100 2026-02-17T05:35:41.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:41.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj1 testdata.31194 2026-02-17T05:35:41.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:41.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj2 testdata.31194 2026-02-17T05:35:41.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:41.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj3 testdata.31194 2026-02-17T05:35:41.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:41.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj4 testdata.31194 2026-02-17T05:35:41.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:41.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj5 testdata.31194 2026-02-17T05:35:41.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:41.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj6 testdata.31194 2026-02-17T05:35:41.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:41.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj7 testdata.31194 2026-02-17T05:35:42.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj8 testdata.31194 2026-02-17T05:35:42.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj9 testdata.31194 2026-02-17T05:35:42.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj10 testdata.31194 2026-02-17T05:35:42.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj11 testdata.31194 2026-02-17T05:35:42.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj12 testdata.31194 2026-02-17T05:35:42.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj13 testdata.31194 2026-02-17T05:35:42.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj14 testdata.31194 2026-02-17T05:35:42.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj15 testdata.31194 2026-02-17T05:35:42.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj16 testdata.31194 2026-02-17T05:35:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj17 testdata.31194 2026-02-17T05:35:42.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj18 testdata.31194 2026-02-17T05:35:42.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj19 testdata.31194 2026-02-17T05:35:42.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:42.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj20 testdata.31194 2026-02-17T05:35:43.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj21 testdata.31194 2026-02-17T05:35:43.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj22 testdata.31194 2026-02-17T05:35:43.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj23 testdata.31194 2026-02-17T05:35:43.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj24 testdata.31194 2026-02-17T05:35:43.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj25 testdata.31194 2026-02-17T05:35:43.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj26 testdata.31194 2026-02-17T05:35:43.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj27 testdata.31194 2026-02-17T05:35:43.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj28 testdata.31194 2026-02-17T05:35:43.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj29 testdata.31194 2026-02-17T05:35:43.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj30 testdata.31194 2026-02-17T05:35:43.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj31 testdata.31194 2026-02-17T05:35:43.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:43.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj32 testdata.31194 2026-02-17T05:35:44.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj33 testdata.31194 2026-02-17T05:35:44.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj34 testdata.31194 2026-02-17T05:35:44.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj35 testdata.31194 2026-02-17T05:35:44.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj36 testdata.31194 2026-02-17T05:35:44.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj37 testdata.31194 2026-02-17T05:35:44.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj38 testdata.31194 2026-02-17T05:35:44.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj39 testdata.31194 2026-02-17T05:35:44.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj40 testdata.31194 2026-02-17T05:35:44.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj41 testdata.31194 2026-02-17T05:35:44.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj42 testdata.31194 2026-02-17T05:35:44.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj43 testdata.31194 2026-02-17T05:35:44.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:44.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj44 testdata.31194 2026-02-17T05:35:45.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj45 testdata.31194 2026-02-17T05:35:45.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj46 testdata.31194 2026-02-17T05:35:45.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj47 testdata.31194 2026-02-17T05:35:45.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj48 testdata.31194 2026-02-17T05:35:45.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj49 testdata.31194 2026-02-17T05:35:45.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj50 testdata.31194 2026-02-17T05:35:45.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj51 testdata.31194 2026-02-17T05:35:45.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj52 testdata.31194 2026-02-17T05:35:45.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj53 testdata.31194 2026-02-17T05:35:45.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj54 testdata.31194 2026-02-17T05:35:45.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj55 testdata.31194 2026-02-17T05:35:45.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:45.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj56 testdata.31194 2026-02-17T05:35:46.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj57 testdata.31194 2026-02-17T05:35:46.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj58 testdata.31194 2026-02-17T05:35:46.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj59 testdata.31194 2026-02-17T05:35:46.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj60 testdata.31194 2026-02-17T05:35:46.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj61 testdata.31194 2026-02-17T05:35:46.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj62 testdata.31194 2026-02-17T05:35:46.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj63 testdata.31194 2026-02-17T05:35:46.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj64 testdata.31194 2026-02-17T05:35:46.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj65 testdata.31194 2026-02-17T05:35:46.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj66 testdata.31194 2026-02-17T05:35:46.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj67 testdata.31194 2026-02-17T05:35:46.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:46.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj68 testdata.31194 2026-02-17T05:35:47.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj69 testdata.31194 2026-02-17T05:35:47.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj70 testdata.31194 2026-02-17T05:35:47.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj71 testdata.31194 2026-02-17T05:35:47.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj72 testdata.31194 2026-02-17T05:35:47.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj73 testdata.31194 2026-02-17T05:35:47.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj74 testdata.31194 2026-02-17T05:35:47.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj75 testdata.31194 2026-02-17T05:35:47.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj76 testdata.31194 2026-02-17T05:35:47.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj77 testdata.31194 2026-02-17T05:35:47.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj78 testdata.31194 2026-02-17T05:35:47.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:47.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj79 testdata.31194 2026-02-17T05:35:48.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj80 testdata.31194 2026-02-17T05:35:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj81 testdata.31194 2026-02-17T05:35:48.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj82 testdata.31194 2026-02-17T05:35:48.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj83 testdata.31194 2026-02-17T05:35:48.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj84 testdata.31194 2026-02-17T05:35:48.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj85 testdata.31194 2026-02-17T05:35:48.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj86 testdata.31194 2026-02-17T05:35:48.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj87 testdata.31194 2026-02-17T05:35:48.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj88 testdata.31194 2026-02-17T05:35:48.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj89 testdata.31194 2026-02-17T05:35:48.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj90 testdata.31194 2026-02-17T05:35:48.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:48.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj91 testdata.31194 2026-02-17T05:35:49.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj92 testdata.31194 2026-02-17T05:35:49.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj93 testdata.31194 2026-02-17T05:35:49.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj94 testdata.31194 2026-02-17T05:35:49.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj95 testdata.31194 2026-02-17T05:35:49.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj96 testdata.31194 2026-02-17T05:35:49.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj97 testdata.31194 2026-02-17T05:35:49.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj98 testdata.31194 2026-02-17T05:35:49.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj99 testdata.31194 2026-02-17T05:35:49.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T05:35:49.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj100 testdata.31194 2026-02-17T05:35:49.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:72: TEST_recovery_scrub_1: rm -f testdata.31194 2026-02-17T05:35:49.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:74: TEST_recovery_scrub_1: ceph osd pool set test size 4 2026-02-17T05:35:50.046 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 size to 4 2026-02-17T05:35:50.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:77: TEST_recovery_scrub_1: set -o pipefail 2026-02-17T05:35:50.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:78: TEST_recovery_scrub_1: count=0 2026-02-17T05:35:50.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T05:35:50.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T05:35:50.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T05:35:50.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T05:35:50.196 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:50.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T05:35:52.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 0 -eq 10 2026-02-17T05:35:52.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 0 + 1 2026-02-17T05:35:52.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=1 2026-02-17T05:35:52.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T05:35:52.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T05:35:52.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T05:35:52.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T05:35:52.352 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:52.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T05:35:54.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 1 -eq 10 2026-02-17T05:35:54.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 1 + 1 2026-02-17T05:35:54.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=2 2026-02-17T05:35:54.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T05:35:54.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T05:35:54.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T05:35:54.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T05:35:54.522 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:54.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T05:35:56.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 2 -eq 10 2026-02-17T05:35:56.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 2 + 1 2026-02-17T05:35:56.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=3 2026-02-17T05:35:56.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T05:35:56.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T05:35:56.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T05:35:56.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T05:35:56.683 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:56.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T05:35:58.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 3 -eq 10 2026-02-17T05:35:58.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 3 + 1 2026-02-17T05:35:58.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=4 2026-02-17T05:35:58.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T05:35:58.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T05:35:58.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T05:35:58.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T05:35:58.842 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:58.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:84: TEST_recovery_scrub_1: break 2026-02-17T05:35:58.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:94: TEST_recovery_scrub_1: set +o pipefail 2026-02-17T05:35:58.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:95: TEST_recovery_scrub_1: ceph pg dump pgs 2026-02-17T05:35:59.001 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:35:59.001 INFO:tasks.workunit.client.0.trial074.stdout:1.0 100 0 234 0 0 5242880000 0 0 1300 0 1300 active+recovering+undersized+degraded+remapped 2026-02-17T05:35:51.667916+0000 20'1300 23:1410 [1,0,2,3] 1 [1,0] 1 0'0 2026-02-17T05:35:35.986597+0000 0'0 2026-02-17T05:35:35.986597+0000 0 0 no scrub is scheduled 0 0 2026-02-17T05:35:59.001 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:35:59.001 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:35:59.002 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:35:59.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:97: TEST_recovery_scrub_1: sleep 10 2026-02-17T05:36:09.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:99: TEST_recovery_scrub_1: kill_daemons td/osd-recovery-scrub 2026-02-17T05:36:09.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:36:09.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:36:09.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:36:09.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:36:09.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:36:17.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:36:17.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:101: TEST_recovery_scrub_1: declare -a err_strings 2026-02-17T05:36:17.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:102: TEST_recovery_scrub_1: err_strings[0]='recovery in progress. Only high priority scrubs allowed.' 2026-02-17T05:36:17.331 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: expr 4 - 1 2026-02-17T05:36:17.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: seq 0 3 2026-02-17T05:36:17.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.0.log 2026-02-17T05:36:17.338 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:52.623+0000 7f147f724640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.338 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:57.539+0000 7f147f724640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.338 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:59.599+0000 7f147f724640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.338 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:00.599+0000 7f147f724640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.338 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:02.603+0000 7f147f724640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.339 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:06.495+0000 7f147f724640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.339 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:11.484+0000 7f147f724640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.1.log 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:52.415+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:54.459+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:55.439+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:56.471+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:57.475+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:00.507+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:08.568+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.351 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:10.544+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.352 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:12.504+0000 7f495aa1e640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.2.log 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:51.931+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:53.903+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:56.959+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:04.987+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:05.951+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:06.979+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:09.016+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:10.008+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.354 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:12.036+0000 7f1f486d8640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.3.log 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:35:52.439+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:00.351+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:01.399+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:02.367+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:04.307+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:06.263+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:07.212+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.358 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:36:09.224+0000 7f1783eb4640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T05:36:17.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:108: TEST_recovery_scrub_1: for err_string in "${err_strings[@]}" 2026-02-17T05:36:17.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:110: TEST_recovery_scrub_1: found=false 2026-02-17T05:36:17.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:111: TEST_recovery_scrub_1: count=0 2026-02-17T05:36:17.359 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: expr 4 - 1 2026-02-17T05:36:17.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: seq 0 3 2026-02-17T05:36:17.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.0.log 2026-02-17T05:36:17.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T05:36:17.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 0 + 1 2026-02-17T05:36:17.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=1 2026-02-17T05:36:17.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.1.log 2026-02-17T05:36:17.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T05:36:17.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 1 + 1 2026-02-17T05:36:17.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=2 2026-02-17T05:36:17.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.2.log 2026-02-17T05:36:17.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T05:36:17.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 2 + 1 2026-02-17T05:36:17.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=3 2026-02-17T05:36:17.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T05:36:17.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.3.log 2026-02-17T05:36:17.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T05:36:17.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 3 + 1 2026-02-17T05:36:17.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=4 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:120: TEST_recovery_scrub_1: '[' true = false ']' 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:124: TEST_recovery_scrub_1: '[' 4 -eq 4 ']' 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:127: TEST_recovery_scrub_1: teardown td/osd-recovery-scrub 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:36:17.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:36:17.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:36:17.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:36:17.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:36:17.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:36:17.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:36:17.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:36:17.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:36:17.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:36:17.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:36:17.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:36:17.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:36:17.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2026-02-17T05:36:17.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:36:17.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:36:17.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.31194 2026-02-17T05:36:17.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:36:17.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:36:17.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:129: TEST_recovery_scrub_1: '[' 0 '!=' 0 ']' 2026-02-17T05:36:17.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:135: TEST_recovery_scrub_1: echo 'TEST PASSED' 2026-02-17T05:36:17.441 INFO:tasks.workunit.client.0.trial074.stdout:TEST PASSED 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:136: TEST_recovery_scrub_1: return 0 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-recovery-scrub 0 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:36:17.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:36:17.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:36:17.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:36:17.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:36:17.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:36:17.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:36:17.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:36:17.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:36:17.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:36:17.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:36:17.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:36:17.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:36:17.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:36:17.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:36:17.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T05:36:17.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2026-02-17T05:36:17.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:36:17.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31194 2026-02-17T05:36:17.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.31194 2026-02-17T05:36:17.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:36:17.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:36:17.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T05:36:17.453 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T05:36:17.453 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T05:36:17.503 INFO:tasks.workunit:Running workunit scrub/osd-scrub-dump.sh... 2026-02-17T05:36:17.503 DEBUG:teuthology.orchestra.run.trial074:workunit test scrub/osd-scrub-dump.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh 2026-02-17T05:36:17.553 INFO:tasks.workunit.client.0.trial074.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T05:36:17.555 INFO:tasks.workunit.client.0.trial074.stdout:This test is disabled 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-dump 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:30: run: echo 'This test is disabled' 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:31: run: return 0 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-dump 0 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-dump 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-dump KILL 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:36:17.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:36:17.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:36:17.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:36:17.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:36:17.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:36:17.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:36:17.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:36:17.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:36:17.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:36:17.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:36:17.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:36:17.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T05:36:17.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-dump 2026-02-17T05:36:17.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:36:17.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40465 2026-02-17T05:36:17.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40465 2026-02-17T05:36:17.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:36:17.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:36:17.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T05:36:17.568 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T05:36:17.568 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T05:36:17.620 INFO:tasks.workunit:Running workunit scrub/osd-scrub-repair.sh... 2026-02-17T05:36:17.620 DEBUG:teuthology.orchestra.run.trial074:workunit test scrub/osd-scrub-repair.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ TIMEOUT=300 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ MAX_TIMEOUT=15 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ PG_NUM=4 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ TMPDIR=/tmp 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ type xmlstarlet 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:++ XMLSTARLET=xmlstarlet 2026-02-17T05:36:17.671 INFO:tasks.workunit.client.0.trial074.stderr:+++ uname 2026-02-17T05:36:17.672 INFO:tasks.workunit.client.0.trial074.stderr:++ '[' Linux = FreeBSD ']' 2026-02-17T05:36:17.672 INFO:tasks.workunit.client.0.trial074.stderr:++ SED=sed 2026-02-17T05:36:17.672 INFO:tasks.workunit.client.0.trial074.stderr:++ AWK=awk 2026-02-17T05:36:17.672 INFO:tasks.workunit.client.0.trial074.stderr:+++ stty -a 2026-02-17T05:36:17.672 INFO:tasks.workunit.client.0.trial074.stderr:+++ head -1 2026-02-17T05:36:17.672 INFO:tasks.workunit.client.0.trial074.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-17T05:36:17.673 INFO:tasks.workunit.client.0.trial074.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T05:36:17.673 INFO:tasks.workunit.client.0.trial074.stderr:++ termwidth= 2026-02-17T05:36:17.673 INFO:tasks.workunit.client.0.trial074.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-17T05:36:17.673 INFO:tasks.workunit.client.0.trial074.stderr:++ DIFFCOLOPTS='-y ' 2026-02-17T05:36:17.674 INFO:tasks.workunit.client.0.trial074.stderr:++ KERNCORE=kernel.core_pattern 2026-02-17T05:36:17.674 INFO:tasks.workunit.client.0.trial074.stderr:++ EXTRA_OPTS= 2026-02-17T05:36:17.675 INFO:tasks.workunit.client.0.trial074.stderr:++ test '' = TESTS 2026-02-17T05:36:17.675 INFO:tasks.workunit.client.0.trial074.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh 2026-02-17T05:36:17.676 INFO:tasks.workunit.client.0.trial074.stderr:++ uname 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:+ '[' Linux = FreeBSD ']' 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:+ use_ec_overwrite=true 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:+ getjson=no 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:+ jqfilter='def walk(f): 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T05:36:17.677 INFO:tasks.workunit.client.0.trial074.stderr:+ sortkeys='import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T05:36:17.681 INFO:tasks.workunit.client.0.trial074.stderr:+ main osd-scrub-repair 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:+ local dir=td/osd-scrub-repair 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:+ shift 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:+ shopt -s -o xtrace 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-repair 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:52: run: local dir=td/osd-scrub-repair 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:53: run: shift 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:55: run: export CEPH_MON=127.0.0.1:7107 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:55: run: CEPH_MON=127.0.0.1:7107 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:56: run: export CEPH_ARGS 2026-02-17T05:36:17.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:57: run: uuidgen 2026-02-17T05:36:17.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:57: run: CEPH_ARGS+='--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none ' 2026-02-17T05:36:17.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:58: run: CEPH_ARGS+='--mon-host=127.0.0.1:7107 ' 2026-02-17T05:36:17.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:59: run: CEPH_ARGS+='--osd-skip-data-digest=false ' 2026-02-17T05:36:17.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:61: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T05:36:17.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: set 2026-02-17T05:36:17.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: local 'funcs=TEST_allow_repair_during_recovery 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_auto_repair_bluestore_basic 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_auto_repair_bluestore_failed 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_auto_repair_bluestore_failed_norecov 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_auto_repair_bluestore_scrub 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_auto_repair_bluestore_tag 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_auto_repair_erasure_coded_appends 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_auto_repair_erasure_coded_overwrites 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_and_repair_jerasure_appends 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_and_repair_jerasure_overwrites 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_and_repair_lrc_appends 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_and_repair_lrc_overwrites 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_and_repair_replicated 2026-02-17T05:36:17.687 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_scrub_erasure_appends 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_scrub_erasure_overwrites 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_scrub_replicated 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_corrupt_snapset_scrub_rep 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_list_missing_erasure_coded_appends 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_list_missing_erasure_coded_overwrites 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_periodic_scrub_replicated 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_repair_stats 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_repair_stats_ec 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_request_scrub_priority 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_scrub_warning 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_skip_non_repair_during_recovery 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_unfound_erasure_coded_appends 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:TEST_unfound_erasure_coded_overwrites' 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:36:17.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:36:17.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:36:17.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:36:17.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:36:17.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:36:17.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:36:17.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:36:17.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:36:17.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:36:17.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:36:17.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:36:17.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:36:17.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:36:17.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:36:17.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:36:17.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:36:17.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:36:17.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:36:17.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:36:17.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:36:17.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:17.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:36:17.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:36:17.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:36:17.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:36:17.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:36:17.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:17.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:36:17.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_allow_repair_during_recovery td/osd-scrub-repair 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:114: TEST_allow_repair_during_recovery: local dir=td/osd-scrub-repair 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:115: TEST_allow_repair_during_recovery: local poolname=rbd 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:117: TEST_allow_repair_during_recovery: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:36:17.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T05:36:17.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:36:17.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:36:17.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:36:17.729 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:36:17.729 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.729 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:17.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:17.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T05:36:17.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:36:17.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:36:17.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:36:17.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:36:17.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:36:17.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:36:17.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:36:17.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:36:17.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:36:17.761 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:36:17.761 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.761 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:17.762 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:36:17.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:36:17.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:36:17.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:36:17.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:36:17.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:36:17.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:36:17.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:36:17.820 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:36:17.820 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:36:17.820 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:36:17.821 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:36:17.821 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.821 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:17.821 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:36:17.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:36:17.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:36:17.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:118: TEST_allow_repair_during_recovery: run_mgr td/osd-scrub-repair x 2026-02-17T05:36:17.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:36:17.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:36:17.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:36:17.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:36:17.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:36:17.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:36:17.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:36:17.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:36:17.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:36:17.968 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:36:17.968 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.968 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:17.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:17.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:36:17.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:36:17.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:119: TEST_allow_repair_during_recovery: run_osd td/osd-scrub-repair 0 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T05:36:17.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:36:17.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:36:17.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:36:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:36:17.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T05:36:17.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:36:17.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:36:17.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=17d4b783-ee9f-4877-907f-6e2720ef8db4 2026-02-17T05:36:17.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 17d4b783-ee9f-4877-907f-6e2720ef8db4' 2026-02-17T05:36:17.994 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 17d4b783-ee9f-4877-907f-6e2720ef8db4 2026-02-17T05:36:17.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:36:18.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBR/pNpVUduOxAA/QNY2dDYwZTorgeKo/Bk6Q== 2026-02-17T05:36:18.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBR/pNpVUduOxAA/QNY2dDYwZTorgeKo/Bk6Q=="}' 2026-02-17T05:36:18.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 17d4b783-ee9f-4877-907f-6e2720ef8db4 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:36:18.089 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:36:18.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:36:18.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQBR/pNpVUduOxAA/QNY2dDYwZTorgeKo/Bk6Q== --osd-uuid 17d4b783-ee9f-4877-907f-6e2720ef8db4 2026-02-17T05:36:18.114 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:18.104+0000 7fce6f8808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:18.115 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:18.108+0000 7fce6f8808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:18.116 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:18.108+0000 7fce6f8808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:18.117 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:18.108+0000 7fce6f8808c0 -1 bdev(0x5565bc701c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:36:18.117 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:18.108+0000 7fce6f8808c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:36:20.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:36:20.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:36:20.989 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:36:20.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:36:20.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:36:21.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:36:21.182 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:36:21.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T05:36:21.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:36:21.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:36:21.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:36:21.207 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:21.197+0000 7f393ee798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:21.209 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:21.201+0000 7f393ee798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:21.210 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:21.201+0000 7f393ee798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:21.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:36:21.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:36:21.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:36:21.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:36:21.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:36:21.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:21.333 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:36:21.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:36:21.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:21.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:21.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:22.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:22.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:22.486 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:36:22.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:36:22.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:22.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:22.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:22.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:22.709+0000 7f393ee798c0 -1 Falling back to public interface 2026-02-17T05:36:23.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:23.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:23.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:36:23.638 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:36:23.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:23.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:23.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:24.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:24.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:24.790 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:36:24.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:36:24.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:24.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:24.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:25.753 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:25.745+0000 7f393ee798c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:36:25.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:25.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:25.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:36:25.946 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:36:25.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:25.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:26.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:27.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:27.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:27.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:36:27.096 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:36:27.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:27.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:27.248 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/234783691,v1:127.0.0.1:6803/234783691] [v2:127.0.0.1:6804/234783691,v1:127.0.0.1:6805/234783691] exists,up 17d4b783-ee9f-4877-907f-6e2720ef8db4 2026-02-17T05:36:27.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:36:27.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:122: TEST_allow_repair_during_recovery: run_osd td/osd-scrub-repair 1 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:36:27.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:27.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T05:36:27.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:36:27.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:36:27.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2444b1c7-731d-4f44-8daa-56d953daf9c3 2026-02-17T05:36:27.256 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 2444b1c7-731d-4f44-8daa-56d953daf9c3 2026-02-17T05:36:27.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 2444b1c7-731d-4f44-8daa-56d953daf9c3' 2026-02-17T05:36:27.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:36:27.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBb/pNpU1SfDxAAy526zae6HSal7cz84F7+gQ== 2026-02-17T05:36:27.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBb/pNpU1SfDxAAy526zae6HSal7cz84F7+gQ=="}' 2026-02-17T05:36:27.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2444b1c7-731d-4f44-8daa-56d953daf9c3 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:36:27.422 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:36:27.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:36:27.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQBb/pNpU1SfDxAAy526zae6HSal7cz84F7+gQ== --osd-uuid 2444b1c7-731d-4f44-8daa-56d953daf9c3 2026-02-17T05:36:27.449 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:27.441+0000 7efced8988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:27.450 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:27.441+0000 7efced8988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:27.451 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:27.445+0000 7efced8988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:27.451 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:27.445+0000 7efced8988c0 -1 bdev(0x55a045ea1c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:36:27.452 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:27.445+0000 7efced8988c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:36:29.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:36:29.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:36:29.732 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:36:29.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:36:29.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:36:29.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:36:29.919 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:36:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T05:36:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:36:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:36:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:36:29.936 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:29.929+0000 7fc1459508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:29.938 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:29.929+0000 7fc1459508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:29.939 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:29.933+0000 7fc1459508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:30.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:36:30.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:31.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:31.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:31.222 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:36:31.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:36:31.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:31.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:36:31.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:31.428 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:31.421+0000 7fc1459508c0 -1 Falling back to public interface 2026-02-17T05:36:32.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:32.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:32.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:36:32.375 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:36:32.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:32.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:36:32.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:33.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:33.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:33.527 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:36:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:36:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:36:33.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:34.390 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:34.381+0000 7fc1459508c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:36:34.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:34.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:34.683 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:36:34.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:36:34.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:34.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:36:34.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:35.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:35.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:35.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:36:35.835 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:36:35.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:35.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:36:35.988 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2550889349,v1:127.0.0.1:6811/2550889349] [v2:127.0.0.1:6812/2550889349,v1:127.0.0.1:6813/2550889349] exists,up 2444b1c7-731d-4f44-8daa-56d953daf9c3 2026-02-17T05:36:35.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:36:35.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:36:35.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:36:35.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:125: TEST_allow_repair_during_recovery: create_rbd_pool 2026-02-17T05:36:35.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:36:36.131 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:36:36.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:36:36.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:36:36.384 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:36:36.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:36:37.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:36:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:126: TEST_allow_repair_during_recovery: wait_for_clean 2026-02-17T05:36:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:36:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:36:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:36:37.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:36:37.659 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:36:37.660 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:36:37.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:36:37.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:36:37.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:36:37.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:36:37.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:36:37.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:36:37.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:36:37.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:36:37.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:36:37.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:36:37.920 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T05:36:37.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:36:37.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:36:37.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:36:37.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T05:36:37.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T05:36:37.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T05:36:37.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:36:37.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:36:38.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705667 2026-02-17T05:36:38.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705667 2026-02-17T05:36:38.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-17T05:36:38.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:36:38.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T05:36:38.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:36:38.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:36:38.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T05:36:38.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:36:38.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T05:36:38.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T05:36:38.067 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T05:36:38.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:36:38.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T05:36:38.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:36:39.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:36:39.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:36:39.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T05:36:39.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:36:39.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705667 2026-02-17T05:36:39.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:36:39.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:36:39.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705667 2026-02-17T05:36:39.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:36:39.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705667 2026-02-17T05:36:39.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-17T05:36:39.383 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705667 2026-02-17T05:36:39.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:36:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-17T05:36:39.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:36:39.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:36:39.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:36:39.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:36:39.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:36:39.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:36:39.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:36:39.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:36:39.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:36:39.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:36:39.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:36:39.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:36:39.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:36:39.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:36:39.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:128: TEST_allow_repair_during_recovery: add_something td/osd-scrub-repair rbd 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T05:36:40.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T05:36:40.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T05:36:40.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T05:36:40.240 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T05:36:40.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T05:36:40.443 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T05:36:40.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T05:36:40.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T05:36:40.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:36:40.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:129: TEST_allow_repair_during_recovery: get_not_primary rbd SOMETHING 2026-02-17T05:36:40.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=rbd 2026-02-17T05:36:40.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:36:40.482 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary rbd SOMETHING 2026-02-17T05:36:40.482 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T05:36:40.482 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:36:40.482 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T05:36:40.482 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:36:40.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:36:40.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T05:36:40.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:36:40.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:129: TEST_allow_repair_during_recovery: corrupt_and_repair_one td/osd-scrub-repair rbd 0 2026-02-17T05:36:40.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:36:40.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=rbd 2026-02-17T05:36:40.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=0 2026-02-17T05:36:40.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:36:40.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:36:42.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:36:42.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:36:42.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:36:42.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:36:42.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:36:42.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:36:42.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:36:42.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T05:36:42.735 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:36:43.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:36:43.273 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:36:43.273 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:43.273 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:43.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:36:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:36:43.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:36:43.275 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:36:43.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:36:43.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:36:43.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:36:43.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:36:43.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:36:43.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:36:43.294 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:43.285+0000 7f36efa7f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:43.295 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:43.289+0000 7f36efa7f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:43.296 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:43.289+0000 7f36efa7f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:43.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:43.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:44.524 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:44.517+0000 7f36efa7f8c0 -1 Falling back to public interface 2026-02-17T05:36:44.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:44.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:44.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:36:44.579 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:36:44.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:44.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:44.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:45.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:45.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:45.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:36:45.727 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:36:45.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:45.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:45.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:46.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:46.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:46.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:36:46.882 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:36:46.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:46.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:47.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:36:47.556 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:47.549+0000 7f36efa7f8c0 -1 osd.0 18 log_to_monitors true 2026-02-17T05:36:48.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:36:48.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:48.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:36:48.031 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:36:48.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:48.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:36:48.181 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 23 up_thru 23 down_at 19 last_clean_interval [5,18) [v2:127.0.0.1:6802/2319902070,v1:127.0.0.1:6803/2319902070] [v2:127.0.0.1:6804/2319902070,v1:127.0.0.1:6805/2319902070] exists,up 17d4b783-ee9f-4877-907f-6e2720ef8db4 2026-02-17T05:36:48.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:36:48.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:36:48.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:36:48.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:36:48.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:36:48.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:36:48.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:36:48.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:36:48.183 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:36:48.183 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:36:48.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:36:48.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:36:48.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:36:48.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:36:48.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:36:48.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:36:48.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:36:48.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:36:48.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:36:48.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:36:48.443 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T05:36:48.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:36:48.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:36:48.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:36:48.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T05:36:48.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T05:36:48.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T05:36:48.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:36:48.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:36:48.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T05:36:48.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T05:36:48.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705670' 2026-02-17T05:36:48.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:36:48.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T05:36:48.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:36:48.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:36:48.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T05:36:48.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:36:48.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T05:36:48.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T05:36:48.592 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 98784247810 2026-02-17T05:36:48.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:36:48.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 98784247810 2026-02-17T05:36:48.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:36:49.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:36:49.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:36:49.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 98784247810 2026-02-17T05:36:49.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:36:50.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:36:50.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:36:51.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T05:36:51.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:36:51.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T05:36:51.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:36:51.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:36:51.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T05:36:51.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:36:51.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T05:36:51.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T05:36:51.061 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705670 2026-02-17T05:36:51.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:36:51.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T05:36:51.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:36:51.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:36:51.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:36:51.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:36:51.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:36:51.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:36:51.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:36:51.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:36:51.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:36:51.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:36:51.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:36:51.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:36:51.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:36:51.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:36:51.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T05:36:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:36:51.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=1.3 2026-02-17T05:36:51.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 1.3 2026-02-17T05:36:51.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.3 2026-02-17T05:36:51.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.3 2026-02-17T05:36:51.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:36:51.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:36:51.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:36:51.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:36:52.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:52.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.3 2026-02-17T05:36:52.169 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-17T05:36:52.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.3 2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:52.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T05:36:52.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:52.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:36:52.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:36:52.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:36:52.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:36:52.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:36:52.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:36:52.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:36:52.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:36:52.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:36:36.380592+0000 '>' 2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:52.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:36:53.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:36:53.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:36:53.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:36:53.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:36:53.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:36:53.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:36:53.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:36:53.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:36:36.380592+0000 '>' 2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:53.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:36:54.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:36:54.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:36:54.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:36:54.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:36:54.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:36:54.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:36:54.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:36:54.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:36:36.380592+0000 '>' 2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:54.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:36:55.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:36:55.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:36:55.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:36:55.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:36:55.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:36:55.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:36:55.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:36:55.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:36:36.380592+0000 '>' 2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:55.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:36:56.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:36:56.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:36:56.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:36:56.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:36:56.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:36:56.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:36:56.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:36:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:36:52.293750+0000 '>' 2026-02-17T05:36:36.380592+0000 2026-02-17T05:36:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:36:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:36:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:36:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:36:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:36:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:36:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:36:58.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:36:58.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:36:58.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:36:58.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:36:58.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:36:58.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:36:58.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:36:58.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:36:58.574 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T05:36:58.574 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:36:58.574 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:36:59.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:36:59.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:36:59.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:36:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:36:59.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:36:59.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:36:59.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:36:59.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:36:59.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:36:59.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:36:59.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:36:59.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:36:59.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:36:59.122 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:36:59.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:36:59.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:36:59.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:36:59.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:36:59.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:36:59.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:36:59.137 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:59.129+0000 7f149e3f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:59.138 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:59.133+0000 7f149e3f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:59.139 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:36:59.133+0000 7f149e3f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:36:59.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:36:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:36:59.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:00.400 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:00.393+0000 7f149e3f88c0 -1 Falling back to public interface 2026-02-17T05:37:00.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:00.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:00.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:37:00.426 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:37:00.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:00.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:37:00.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:01.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:01.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:01.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:37:01.574 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:37:01.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:01.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:37:01.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:02.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:02.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:02.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:37:02.724 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:37:02.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:02.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:37:02.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:03.409 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:03.401+0000 7f149e3f88c0 -1 osd.1 26 log_to_monitors true 2026-02-17T05:37:03.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:03.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:03.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:37:03.871 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:37:03.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:03.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:37:04.020 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 30 up_thru 30 down_at 27 last_clean_interval [9,26) [v2:127.0.0.1:6810/877697919,v1:127.0.0.1:6811/877697919] [v2:127.0.0.1:6812/877697919,v1:127.0.0.1:6813/877697919] exists,up 2444b1c7-731d-4f44-8daa-56d953daf9c3 2026-02-17T05:37:04.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:37:04.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:37:04.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:37:04.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:37:04.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:37:04.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:37:04.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:37:04.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:37:04.021 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:37:04.021 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:37:04.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:37:04.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:37:04.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:37:04.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:37:04.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:37:04.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:37:04.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:37:04.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:37:04.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:37:04.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:37:04.268 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T05:37:04.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:37:04.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:04.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:37:04.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247814 2026-02-17T05:37:04.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247814 2026-02-17T05:37:04.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247814' 2026-02-17T05:37:04.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:04.340 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:37:04.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T05:37:04.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T05:37:04.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247814 1-128849018882' 2026-02-17T05:37:04.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:04.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247814 2026-02-17T05:37:04.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:04.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:37:04.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247814 2026-02-17T05:37:04.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:04.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247814 2026-02-17T05:37:04.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247814' 2026-02-17T05:37:04.417 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 98784247814 2026-02-17T05:37:04.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:04.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247813 -lt 98784247814 2026-02-17T05:37:04.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:37:05.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:37:05.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:05.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247813 -lt 98784247814 2026-02-17T05:37:05.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:37:06.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:37:06.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:06.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247814 -lt 98784247814 2026-02-17T05:37:06.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:06.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-128849018882 2026-02-17T05:37:06.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:06.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:37:06.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-128849018882 2026-02-17T05:37:06.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:06.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T05:37:06.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 128849018882' 2026-02-17T05:37:06.884 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 128849018882 2026-02-17T05:37:06.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:37:07.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-17T05:37:07.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:37:07.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:07.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:07.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:37:07.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:37:07.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:37:07.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:37:07.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:37:07.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:37:07.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:37:07.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:37:07.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:37:07.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:37:07.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:07.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:07.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:37:07.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:37:07.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:37:07.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:37:07.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:37:07.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:37:07.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:37:07.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:37:07.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:37:07.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:37:07.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:37:07.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:37:07.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:37:07.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:37:07.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:37:07.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:37:07.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:37:07.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:37:07.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:37:07.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:37:07.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:37:07.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:37:07.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:37:07.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:37:07.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:37:07.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:37:07.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:37:07.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:37:07.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:37:07.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:37:07.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:37:07.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:37:07.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:37:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:37:07.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:37:07.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:37:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:37:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:37:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:37:07.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:37:07.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:37:07.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:37:07.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:37:07.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:37:07.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:37:07.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:37:07.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:37:07.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:37:07.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:37:07.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:37:07.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:37:07.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:37:07.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:37:07.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:37:07.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:37:07.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:37:07.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:37:07.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:37:07.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:37:07.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:37:07.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:37:07.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:37:07.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:37:07.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:37:07.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:37:07.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:37:07.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:37:07.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:37:07.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_basic td/osd-scrub-repair 2026-02-17T05:37:07.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:438: TEST_auto_repair_bluestore_basic: local dir=td/osd-scrub-repair 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:439: TEST_auto_repair_bluestore_basic: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='1' ['pool_name']='testpool' ['extras']=' --osd_scrub_auto_repair=true') 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:439: TEST_auto_repair_bluestore_basic: local -A cluster_conf 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:445: TEST_auto_repair_bluestore_basic: local extr_dbg=3 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:446: TEST_auto_repair_bluestore_basic: standard_scrub_cluster td/osd-scrub-repair cluster_conf 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-repair 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=1 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:234: standard_scrub_cluster: local poolname=testpool 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: args['pool_name']=testpool 2026-02-17T05:37:07.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: local 'extra_pars= --osd_scrub_auto_repair=true' 2026-02-17T05:37:07.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:237: standard_scrub_cluster: local debug_msg=dbg 2026-02-17T05:37:07.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:240: standard_scrub_cluster: local saved_echo_flag=x 2026-02-17T05:37:07.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: set +x 2026-02-17T05:37:08.000 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 b44dd247-1760-4676-a5b9-8113a239f8de 2026-02-17T05:37:08.101 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:37:08.128 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:08.122+0000 7fc6f63e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:08.130 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:08.122+0000 7fc6f63e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:08.130 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:08.126+0000 7fc6f63e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:08.131 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:08.126+0000 7fc6f63e08c0 -1 bdev(0x5602dc599c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:37:08.131 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:08.126+0000 7fc6f63e08c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:37:10.935 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:37:11.127 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:37:11.153 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:11.146+0000 7f9f8d59d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:11.155 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:11.150+0000 7f9f8d59d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:11.156 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:11.150+0000 7f9f8d59d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:11.278 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:37:12.430 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:37:12.639 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:12.634+0000 7f9f8d59d8c0 -1 Falling back to public interface 2026-02-17T05:37:13.578 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:37:14.727 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:37:15.628 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:15.622+0000 7f9f8d59d8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:37:15.875 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:37:17.029 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:37:17.177 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3035835613,v1:127.0.0.1:6803/3035835613] [v2:127.0.0.1:6804/3035835613,v1:127.0.0.1:6805/3035835613] exists,up b44dd247-1760-4676-a5b9-8113a239f8de 2026-02-17T05:37:17.182 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 5679b8e7-dad2-4675-affb-2b95bfc08724 2026-02-17T05:37:17.352 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:37:17.380 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:17.374+0000 7fc21c7f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:17.381 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:17.378+0000 7fc21c7f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:17.382 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:17.378+0000 7fc21c7f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:17.383 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:17.378+0000 7fc21c7f88c0 -1 bdev(0x55a3704bbc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:37:17.383 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:17.378+0000 7fc21c7f88c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:37:19.655 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:37:19.850 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:37:19.877 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:19.870+0000 7f63513a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:19.878 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:19.874+0000 7f63513a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:19.879 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:19.874+0000 7f63513a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:20.004 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:37:21.155 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:37:21.403 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:21.398+0000 7f63513a58c0 -1 Falling back to public interface 2026-02-17T05:37:22.309 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:37:23.465 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:37:24.362 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:24.358+0000 7f63513a58c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:37:24.615 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:37:25.771 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:37:25.925 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1112122765,v1:127.0.0.1:6811/1112122765] [v2:127.0.0.1:6812/1112122765,v1:127.0.0.1:6813/1112122765] exists,up 5679b8e7-dad2-4675-affb-2b95bfc08724 2026-02-17T05:37:25.930 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 30f7aef9-9297-4a04-aa13-5a3171e4d1cc 2026-02-17T05:37:26.098 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:37:26.124 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:26.118+0000 7f850f31c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:26.126 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:26.122+0000 7f850f31c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:26.127 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:26.122+0000 7f850f31c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:26.127 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:26.122+0000 7f850f31c8c0 -1 bdev(0x55a630befc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:37:26.127 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:26.122+0000 7f850f31c8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:37:28.395 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:37:28.584 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:37:28.612 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:28.606+0000 7fd49f2578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:28.614 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:28.610+0000 7fd49f2578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:28.615 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:28.610+0000 7fd49f2578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:28.740 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:37:29.890 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:37:30.134 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:30.130+0000 7fd49f2578c0 -1 Falling back to public interface 2026-02-17T05:37:31.039 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:37:32.189 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:37:33.160 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:33.155+0000 7fd49f2578c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:37:33.340 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:37:34.493 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:37:34.647 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3818996684,v1:127.0.0.1:6819/3818996684] [v2:127.0.0.1:6820/3818996684,v1:127.0.0.1:6821/3818996684] exists,up 30f7aef9-9297-4a04-aa13-5a3171e4d1cc 2026-02-17T05:37:34.847 INFO:tasks.workunit.client.0.trial074.stderr:pool 'testpool' created 2026-02-17T05:37:36.330 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:37:36.486 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T05:37:38.948 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509442 2026-02-17T05:37:40.748 INFO:tasks.workunit.client.0.trial074.stdout:standard_scrub_cluster: dbg: test pool is testpool 1 2026-02-17T05:37:40.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:447: TEST_auto_repair_bluestore_basic: local poolid=1 2026-02-17T05:37:40.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:448: TEST_auto_repair_bluestore_basic: local poolname=testpool 2026-02-17T05:37:40.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:450: TEST_auto_repair_bluestore_basic: ceph osd pool set testpool size 2 2026-02-17T05:37:40.942 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 size to 2 2026-02-17T05:37:40.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:451: TEST_auto_repair_bluestore_basic: wait_for_clean 2026-02-17T05:37:40.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:37:40.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:37:40.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:37:40.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:37:40.952 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:37:40.952 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:37:40.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:37:40.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:37:40.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:37:41.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:37:41.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:37:41.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:37:41.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:37:41.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:37:41.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:37:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:37:41.217 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:37:41.217 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:37:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:37:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:41.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:37:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T05:37:41.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T05:37:41.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T05:37:41.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:41.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:37:41.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T05:37:41.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T05:37:41.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672966' 2026-02-17T05:37:41.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:41.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:37:41.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T05:37:41.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T05:37:41.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672966 2-64424509444' 2026-02-17T05:37:41.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:41.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T05:37:41.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:37:41.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T05:37:41.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:41.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T05:37:41.435 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T05:37:41.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T05:37:41.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:41.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T05:37:41.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:37:42.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:37:42.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:42.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-17T05:37:42.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:42.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T05:37:42.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:42.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:37:42.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T05:37:42.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:42.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T05:37:42.758 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672966 2026-02-17T05:37:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T05:37:42.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:37:42.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T05:37:42.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:42.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T05:37:42.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:42.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:37:42.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T05:37:42.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:42.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T05:37:42.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T05:37:42.911 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509444 2026-02-17T05:37:42.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:37:43.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T05:37:43.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:37:43.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:43.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:43.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:37:43.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:37:43.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:37:43.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:37:43.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:37:43.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:37:43.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:37:43.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:37:43.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:37:43.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:37:43.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:43.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:43.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:37:43.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:37:43.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:37:43.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:454: TEST_auto_repair_bluestore_basic: local payload=ABCDEF 2026-02-17T05:37:43.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:455: TEST_auto_repair_bluestore_basic: echo ABCDEF 2026-02-17T05:37:43.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:456: TEST_auto_repair_bluestore_basic: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:37:43.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:460: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-17T05:37:43.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:37:43.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:37:43.591 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:37:43.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:37:43.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:37:43.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:37:43.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:37:43.739 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:37:43.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:37:43.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:37:43.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:460: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:37:43.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:37:43.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:37:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:37:43.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:37:43.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:37:45.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T05:37:45.845 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T05:37:46.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T05:37:46.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:37:46.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:37:46.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:37:46.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:37:46.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:37:46.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:37:46.382 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:37:46.382 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:37:46.382 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:37:46.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:37:46.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:37:46.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:37:46.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:37:46.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:37:46.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:37:46.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:37:46.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:37:46.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:37:46.405 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:46.403+0000 7f5d8bbd08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:46.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:46.403+0000 7f5d8bbd08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:46.408 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:46.403+0000 7f5d8bbd08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:37:46.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:37:46.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:37:46.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:37:46.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:37:46.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:37:46.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:46.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:37:46.535 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:37:46.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:46.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:37:46.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:47.658 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:47.655+0000 7f5d8bbd08c0 -1 Falling back to public interface 2026-02-17T05:37:47.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:47.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:47.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:37:47.691 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:37:47.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:47.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:37:47.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:48.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:48.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:48.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:37:48.841 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:37:48.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:48.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:37:48.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:49.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:49.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:49.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:37:49.992 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:37:49.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:49.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:37:50.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:37:50.939 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:37:50.935+0000 7f5d8bbd08c0 -1 osd.0 21 log_to_monitors true 2026-02-17T05:37:51.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:37:51.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:37:51.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:37:51.149 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:37:51.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:37:51.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:37:51.305 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 25 up_thru 0 down_at 22 last_clean_interval [5,21) [v2:127.0.0.1:6802/1828123007,v1:127.0.0.1:6803/1828123007] [v2:127.0.0.1:6804/1828123007,v1:127.0.0.1:6805/1828123007] exists,up b44dd247-1760-4676-a5b9-8113a239f8de 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:37:51.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:37:51.307 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:37:51.307 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:37:51.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:37:51.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:37:51.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:37:51.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:37:51.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:37:51.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:37:51.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:37:51.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:37:51.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:37:51.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:37:51.562 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:37:51.562 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:37:51.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:37:51.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:51.563 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:37:51.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182402 2026-02-17T05:37:51.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182402 2026-02-17T05:37:51.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402' 2026-02-17T05:37:51.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:51.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:37:51.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672969 2026-02-17T05:37:51.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672969 2026-02-17T05:37:51.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402 1-42949672969' 2026-02-17T05:37:51.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:51.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:37:51.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509447 2026-02-17T05:37:51.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509447 2026-02-17T05:37:51.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402 1-42949672969 2-64424509447' 2026-02-17T05:37:51.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:51.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182402 2026-02-17T05:37:51.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:51.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:37:51.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182402 2026-02-17T05:37:51.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:51.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182402 2026-02-17T05:37:51.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182402' 2026-02-17T05:37:51.778 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 107374182402 2026-02-17T05:37:51.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:51.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 107374182402 2026-02-17T05:37:51.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:37:52.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:37:52.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:53.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182402 -lt 107374182402 2026-02-17T05:37:53.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:53.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672969 2026-02-17T05:37:53.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:53.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:37:53.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672969 2026-02-17T05:37:53.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:53.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672969 2026-02-17T05:37:53.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672969' 2026-02-17T05:37:53.082 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672969 2026-02-17T05:37:53.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:37:53.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672969 -lt 42949672969 2026-02-17T05:37:53.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:53.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509447 2026-02-17T05:37:53.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:53.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:37:53.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509447 2026-02-17T05:37:53.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:53.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509447 2026-02-17T05:37:53.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-17T05:37:53.246 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509447 2026-02-17T05:37:53.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:37:53.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509447 -lt 64424509447 2026-02-17T05:37:53.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:37:53.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:53.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:37:53.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:37:53.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:37:53.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:37:53.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:53.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:53.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:37:53.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:37:53.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:37:53.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:461: TEST_auto_repair_bluestore_basic: ceph tell 'osd.*' config set osd_scrub_auto_repair true 2026-02-17T05:37:53.974 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T05:37:53.975 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T05:37:53.975 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:37:53.981 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T05:37:53.981 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-17T05:37:53.981 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:37:53.987 INFO:tasks.workunit.client.0.trial074.stdout:osd.2: { 2026-02-17T05:37:53.987 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-17T05:37:53.987 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:37:53.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:463: TEST_auto_repair_bluestore_basic: get_pg testpool SOMETHING 2026-02-17T05:37:53.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T05:37:53.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:37:53.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-17T05:37:53.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:37:54.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:463: TEST_auto_repair_bluestore_basic: local pgid=1.0 2026-02-17T05:37:54.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:464: TEST_auto_repair_bluestore_basic: get_primary testpool SOMETHING 2026-02-17T05:37:54.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:37:54.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:37:54.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:37:54.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:37:54.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:464: TEST_auto_repair_bluestore_basic: local primary=1 2026-02-17T05:37:54.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:465: TEST_auto_repair_bluestore_basic: get_last_scrub_stamp 1.0 2026-02-17T05:37:54.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:37:54.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:37:54.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:37:54.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:37:54.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:465: TEST_auto_repair_bluestore_basic: local last_scrub_stamp=2026-02-17T05:37:34.846618+0000 2026-02-17T05:37:54.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:468: TEST_auto_repair_bluestore_basic: ceph tell 1.0 schedule-deep-scrub 2026-02-17T05:37:54.513 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:37:54.513 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T05:37:54.513 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T05:37:54.513 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-03T05:36:14.513448+0000" 2026-02-17T05:37:54.513 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:37:54.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:471: TEST_auto_repair_bluestore_basic: wait_for_scrub 1.0 2026-02-17T05:37:34.846618+0000 2026-02-17T05:37:54.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:37:34.846618+0000 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:37:54.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:37:54.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:37:34.846618+0000 '>' 2026-02-17T05:37:34.846618+0000 2026-02-17T05:37:54.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:37:55.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:37:55.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:37:55.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:37:55.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:37:55.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:37:55.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:37:55.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:37:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:37:34.846618+0000 '>' 2026-02-17T05:37:34.846618+0000 2026-02-17T05:37:55.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:37:56.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:37:56.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:37:56.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:37:56.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:37:56.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:37:56.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:37:56.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:37:56.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:37:55.346673+0000 '>' 2026-02-17T05:37:34.846618+0000 2026-02-17T05:37:56.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:37:56.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:472: TEST_auto_repair_bluestore_basic: wait_for_clean 2026-02-17T05:37:56.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:37:56.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:37:56.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:37:56.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:37:56.978 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:37:56.978 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:37:56.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:37:56.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:37:56.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:37:57.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:37:57.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:37:57.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:37:57.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:37:57.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:37:57.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:37:57.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:37:57.236 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:37:57.236 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:37:57.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:37:57.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:57.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:37:57.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182404 2026-02-17T05:37:57.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182404 2026-02-17T05:37:57.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182404' 2026-02-17T05:37:57.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:57.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:37:57.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672971 2026-02-17T05:37:57.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672971 2026-02-17T05:37:57.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182404 1-42949672971' 2026-02-17T05:37:57.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:37:57.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:37:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509449 2026-02-17T05:37:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509449 2026-02-17T05:37:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182404 1-42949672971 2-64424509449' 2026-02-17T05:37:57.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:57.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182404 2026-02-17T05:37:57.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:57.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:37:57.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182404 2026-02-17T05:37:57.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:57.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182404 2026-02-17T05:37:57.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182404' 2026-02-17T05:37:57.454 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 107374182404 2026-02-17T05:37:57.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:57.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182404 2026-02-17T05:37:57.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:37:58.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:37:58.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:37:58.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-17T05:37:58.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:58.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672971 2026-02-17T05:37:58.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:58.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:37:58.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672971 2026-02-17T05:37:58.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:58.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672971 2026-02-17T05:37:58.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-17T05:37:58.768 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672971 2026-02-17T05:37:58.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:37:58.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-17T05:37:58.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:37:58.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509449 2026-02-17T05:37:58.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:37:58.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:37:58.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509449 2026-02-17T05:37:58.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:37:58.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509449 2026-02-17T05:37:58.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-17T05:37:58.932 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509449 2026-02-17T05:37:58.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:37:59.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-17T05:37:59.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:37:59.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:59.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:37:59.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:37:59.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:37:59.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:37:59.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:37:59.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:37:59.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:37:59.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:37:59.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:37:59.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:37:59.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:37:59.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:37:59.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:37:59.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:37:59.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:37:59.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:473: TEST_auto_repair_bluestore_basic: ceph pg dump pgs 2026-02-17T05:37:59.748 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:37:59.748 INFO:tasks.workunit.client.0.trial074.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-17T05:37:55.348846+0000 21'1 26:57 [1,0] 1 [1,0] 1 21'1 2026-02-17T05:37:55.346673+0000 21'1 2026-02-17T05:37:55.346673+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:37:55.346673+0000 1 0 2026-02-17T05:37:59.748 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:37:59.748 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:37:59.748 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:37:59.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:476: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-17T05:37:59.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:37:59.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:37:59.757 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:37:59.757 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:37:59.757 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:37:59.757 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:37:59.758 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:37:59.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:37:59.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:37:59.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:476: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:38:00.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:38:00.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:38:00.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:38:00.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:38:01.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:38:01.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:38:01.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:38:01.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:38:01.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:38:01.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:38:01.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:38:01.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T05:38:01.686 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:38:01.686 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:38:02.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:38:02.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:38:02.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:38:02.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:38:02.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:38:02.224 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:38:02.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:38:02.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:38:02.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:38:02.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:38:02.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:38:02.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:38:02.243 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:02.238+0000 7fdcf90618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:02.245 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:02.242+0000 7fdcf90618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:02.245 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:02.242+0000 7fdcf90618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:02.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:38:02.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:38:02.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:38:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:38:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:38:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:38:02.375 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:38:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:02.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:03.486 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:03.482+0000 7fdcf90618c0 -1 Falling back to public interface 2026-02-17T05:38:03.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:03.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:03.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:38:03.531 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:38:03.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:03.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:03.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:04.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:04.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:04.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:38:04.681 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:38:04.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:04.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:04.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:05.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:05.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:05.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:38:05.836 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:38:05.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:05.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:05.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:06.495 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:06.490+0000 7fdcf90618c0 -1 osd.0 26 log_to_monitors true 2026-02-17T05:38:06.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:06.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:06.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:38:06.989 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:38:06.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:06.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:07.143 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 30 up_thru 0 down_at 27 last_clean_interval [25,26) [v2:127.0.0.1:6802/3217719707,v1:127.0.0.1:6803/3217719707] [v2:127.0.0.1:6804/3217719707,v1:127.0.0.1:6805/3217719707] exists,up b44dd247-1760-4676-a5b9-8113a239f8de 2026-02-17T05:38:07.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:38:07.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:38:07.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:38:07.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:38:07.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:38:07.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:38:07.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:38:07.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:38:07.145 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:38:07.145 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:38:07.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:38:07.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:38:07.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:38:07.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:38:07.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:38:07.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:38:07.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:38:07.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:38:07.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:38:07.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:38:07.406 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:38:07.406 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:38:07.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:38:07.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:07.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:38:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T05:38:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T05:38:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882' 2026-02-17T05:38:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:07.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:38:07.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672974 2026-02-17T05:38:07.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672974 2026-02-17T05:38:07.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-42949672974' 2026-02-17T05:38:07.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:07.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:38:07.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509452 2026-02-17T05:38:07.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509452 2026-02-17T05:38:07.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-42949672974 2-64424509452' 2026-02-17T05:38:07.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:07.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018882 2026-02-17T05:38:07.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:07.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:38:07.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018882 2026-02-17T05:38:07.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:07.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T05:38:07.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018882' 2026-02-17T05:38:07.626 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 128849018882 2026-02-17T05:38:07.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:38:07.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182404 -lt 128849018882 2026-02-17T05:38:07.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:38:08.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:38:08.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:38:08.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-17T05:38:08.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:08.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672974 2026-02-17T05:38:08.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:08.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:38:08.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672974 2026-02-17T05:38:08.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:08.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672974 2026-02-17T05:38:08.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672974' 2026-02-17T05:38:08.945 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672974 2026-02-17T05:38:08.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:38:09.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672974 -lt 42949672974 2026-02-17T05:38:09.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:09.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509452 2026-02-17T05:38:09.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:09.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:38:09.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509452 2026-02-17T05:38:09.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:09.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509452 2026-02-17T05:38:09.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2026-02-17T05:38:09.105 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509452 2026-02-17T05:38:09.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:38:09.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509452 -lt 64424509452 2026-02-17T05:38:09.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:38:09.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:09.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:09.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:38:09.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:09.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:38:09.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:09.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:09.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:09.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:38:09.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:38:09.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:38:09.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:477: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-17T05:38:09.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:38:09.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:38:09.762 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:38:09.763 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:38:09.763 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:38:09.763 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:38:09.763 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:38:09.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:38:09.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:38:09.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:38:10.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:477: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:38:10.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:38:10.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:38:10.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:38:10.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:38:11.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:38:12.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:38:12.234 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:38:12.234 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:12.234 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:12.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:38:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:38:12.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:38:12.236 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:38:12.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:38:12.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:38:12.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:38:12.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:38:12.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:38:12.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:38:12.255 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:12.250+0000 7f795c6ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:12.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:12.254+0000 7f795c6ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:12.258 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:12.254+0000 7f795c6ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:38:12.388 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:38:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:12.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:13.506 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:13.502+0000 7f795c6ab8c0 -1 Falling back to public interface 2026-02-17T05:38:13.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:13.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:13.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:38:13.541 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:38:13.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:13.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:13.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:38:14.693 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:38:14.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:14.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:14.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:15.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:15.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:15.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:38:15.844 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:38:15.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:15.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:15.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:16.473 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:16.470+0000 7f795c6ab8c0 -1 osd.0 31 log_to_monitors true 2026-02-17T05:38:16.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:16.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:16.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:38:16.992 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:38:16.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:16.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:17.140 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 35 up_thru 0 down_at 32 last_clean_interval [30,31) [v2:127.0.0.1:6802/984705455,v1:127.0.0.1:6803/984705455] [v2:127.0.0.1:6804/984705455,v1:127.0.0.1:6805/984705455] exists,up b44dd247-1760-4676-a5b9-8113a239f8de 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:38:17.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:38:17.142 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:38:17.142 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:38:17.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:38:17.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:38:17.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:38:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:38:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:38:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:38:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:38:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:38:17.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:38:17.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:38:17.394 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:38:17.394 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:38:17.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:38:17.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:17.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:38:17.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855362 2026-02-17T05:38:17.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855362 2026-02-17T05:38:17.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362' 2026-02-17T05:38:17.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:17.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:38:17.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672977 2026-02-17T05:38:17.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672977 2026-02-17T05:38:17.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-42949672977' 2026-02-17T05:38:17.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:17.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:38:17.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509455 2026-02-17T05:38:17.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509455 2026-02-17T05:38:17.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-42949672977 2-64424509455' 2026-02-17T05:38:17.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:17.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855362 2026-02-17T05:38:17.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:17.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:38:17.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855362 2026-02-17T05:38:17.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:17.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855362 2026-02-17T05:38:17.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855362' 2026-02-17T05:38:17.615 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 150323855362 2026-02-17T05:38:17.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:38:17.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018882 -lt 150323855362 2026-02-17T05:38:17.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:38:18.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:38:18.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:38:18.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855362 2026-02-17T05:38:18.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:18.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672977 2026-02-17T05:38:18.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:18.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:38:18.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672977 2026-02-17T05:38:18.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:18.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672977 2026-02-17T05:38:18.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672977' 2026-02-17T05:38:18.934 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672977 2026-02-17T05:38:18.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:38:19.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672977 -lt 42949672977 2026-02-17T05:38:19.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:19.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509455 2026-02-17T05:38:19.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:19.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:38:19.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509455 2026-02-17T05:38:19.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:19.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509455 2026-02-17T05:38:19.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509455' 2026-02-17T05:38:19.098 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509455 2026-02-17T05:38:19.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:38:19.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509455 -lt 64424509455 2026-02-17T05:38:19.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:38:19.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:19.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:19.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:38:19.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:19.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:19.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:19.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:19.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:19.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:19.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:19.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:19.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:19.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:19.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T05:38:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:38:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T05:38:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:38:19.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:19.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:19.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:19.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:19.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:19.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:19.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:19.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:20.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:20.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:20.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:20.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:20.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:20.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:38:20.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:38:20.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:38:20.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:38:20.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:38:20.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:38:20.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:38:20.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:38:20.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:38:20.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:38:20.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:38:20.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:20.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:38:20.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:20.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:20.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:20.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:20.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:20.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:20.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:20.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:20.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:20.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:20.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:20.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:20.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:20.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:38:20.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:38:20.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:38:20.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:38:20.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:38:20.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:38:20.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:38:20.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:38:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:38:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:38:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T05:38:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T05:38:21.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:21.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:21.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:21.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:21.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:21.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:21.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:21.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:21.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:21.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:21.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:21.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:38:21.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:38:22.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:38:22.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:38:22.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T05:38:22.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:22.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T05:38:22.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:22.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:22.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:22.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:22.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:22.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:22.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:22.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:22.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:38:22.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:22.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:22.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:23.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:38:23.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:38:23.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:38:23.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:478: TEST_auto_repair_bluestore_basic: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:38:23.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:479: TEST_auto_repair_bluestore_basic: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T05:38:23.136 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:37:55.343+0000 7f633332b640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:38:23.136 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:37:55.343+0000 7f633332b640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T05:38:23.136 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:37:55.343+0000 7f633332b640 15 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T05:38:23.136 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:37:55.343+0000 7f633332b640 20 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-17T05:38:23.136 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:37:55.343+0000 7f633332b640 19 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:38:23.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:38:23.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:38:23.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:38:23.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:38:23.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:38:23.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:38:23.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:38:23.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:38:23.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:38:23.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:38:23.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:38:23.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:38:23.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:38:23.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:38:23.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:38:23.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:38:23.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:38:23.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:38:23.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:38:23.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:38:23.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:38:23.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:38:23.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:38:23.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:38:23.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:38:23.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:38:23.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:38:23.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:38:23.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:38:23.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:38:23.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:38:23.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:38:23.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:38:23.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:38:23.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:38:23.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:38:23.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:38:23.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:38:23.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:38:23.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:38:23.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:38:23.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:38:23.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:38:23.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:38:23.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:38:23.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:38:23.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:38:23.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:38:23.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:38:23.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:38:23.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:38:23.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:38:23.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:38:23.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:38:23.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_failed td/osd-scrub-repair 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:539: TEST_auto_repair_bluestore_failed: local dir=td/osd-scrub-repair 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:540: TEST_auto_repair_bluestore_failed: local poolname=testpool 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:543: TEST_auto_repair_bluestore_failed: run_mon td/osd-scrub-repair a 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:38:23.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:38:23.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:38:23.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:38:23.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:38:23.508 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:38:23.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:23.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:38:23.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:38:23.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:38:23.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:38:23.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:38:23.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:38:23.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:38:23.541 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:38:23.541 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:38:23.541 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:38:23.541 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:38:23.541 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.541 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.542 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:38:23.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:38:23.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:38:23.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:38:23.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:38:23.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:38:23.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:38:23.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:38:23.597 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:38:23.597 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:38:23.597 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:38:23.597 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:38:23.597 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.597 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.598 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:38:23.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:38:23.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:38:23.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:544: TEST_auto_repair_bluestore_failed: run_mgr td/osd-scrub-repair x 2026-02-17T05:38:23.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:38:23.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:38:23.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:38:23.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:38:23.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:38:23.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:38:23.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:38:23.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:38:23.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:38:23.747 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:38:23.747 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.747 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:23.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:38:23.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:38:23.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:547: TEST_auto_repair_bluestore_failed: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:38:23.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: seq 0 2 2026-02-17T05:38:23.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-17T05:38:23.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:549: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:38:23.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:38:23.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:38:23.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:23.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:38:23.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:38:23.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:38:23.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:38:23.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2125a123-d1d1-4e15-9ae1-b0ee620e04fa 2026-02-17T05:38:23.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2125a123-d1d1-4e15-9ae1-b0ee620e04fa' 2026-02-17T05:38:23.778 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 2125a123-d1d1-4e15-9ae1-b0ee620e04fa 2026-02-17T05:38:23.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:38:23.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDP/pNpeqPDLhAASP39WnmvKcZ17AZu3KDa2g== 2026-02-17T05:38:23.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDP/pNpeqPDLhAASP39WnmvKcZ17AZu3KDa2g=="}' 2026-02-17T05:38:23.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2125a123-d1d1-4e15-9ae1-b0ee620e04fa -i td/osd-scrub-repair/0/new.json 2026-02-17T05:38:23.875 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:38:23.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:38:23.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDP/pNpeqPDLhAASP39WnmvKcZ17AZu3KDa2g== --osd-uuid 2125a123-d1d1-4e15-9ae1-b0ee620e04fa 2026-02-17T05:38:23.900 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:23.894+0000 7f44ae5498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:23.902 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:23.898+0000 7f44ae5498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:23.902 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:23.898+0000 7f44ae5498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:23.903 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:23.898+0000 7f44ae5498c0 -1 bdev(0x558c1b325c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:38:23.903 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:23.898+0000 7f44ae5498c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:38:26.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:38:26.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:38:26.524 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:38:26.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:38:26.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:38:26.718 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:38:26.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:38:26.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:38:26.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:38:26.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:38:26.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:38:26.743 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:26.738+0000 7f30bdc2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:26.744 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:26.738+0000 7f30bdc2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:26.745 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:26.742+0000 7f30bdc2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:26.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:27.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:28.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:28.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:28.020 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:38:28.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:38:28.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:28.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:28.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:28.238 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:28.234+0000 7f30bdc2f8c0 -1 Falling back to public interface 2026-02-17T05:38:29.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:29.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:29.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:38:29.176 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:38:29.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:29.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:29.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:30.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:30.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:30.332 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:38:30.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:38:30.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:30.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:30.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:31.228 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:31.222+0000 7f30bdc2f8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:38:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:31.484 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:38:31.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:38:31.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:31.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:31.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:32.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:32.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:32.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:38:32.637 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:38:32.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:32.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:38:32.788 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2624138902,v1:127.0.0.1:6803/2624138902] [v2:127.0.0.1:6804/2624138902,v1:127.0.0.1:6805/2624138902] exists,up 2125a123-d1d1-4e15-9ae1-b0ee620e04fa 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:549: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:38:32.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:38:32.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:38:32.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:38:32.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bbd22585-72ca-44e1-9bd4-763ee2a331f9 2026-02-17T05:38:32.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 bbd22585-72ca-44e1-9bd4-763ee2a331f9' 2026-02-17T05:38:32.793 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 bbd22585-72ca-44e1-9bd4-763ee2a331f9 2026-02-17T05:38:32.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:38:32.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDY/pNpDkD8LxAA/M7cVuDRIifyvhALxFWUSw== 2026-02-17T05:38:32.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDY/pNpDkD8LxAA/M7cVuDRIifyvhALxFWUSw=="}' 2026-02-17T05:38:32.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bbd22585-72ca-44e1-9bd4-763ee2a331f9 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:38:32.962 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:38:32.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:38:32.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDY/pNpDkD8LxAA/M7cVuDRIifyvhALxFWUSw== --osd-uuid bbd22585-72ca-44e1-9bd4-763ee2a331f9 2026-02-17T05:38:32.989 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:32.986+0000 7fd345e3f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:32.991 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:32.986+0000 7fd345e3f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:32.992 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:32.986+0000 7fd345e3f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:32.992 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:32.986+0000 7fd345e3f8c0 -1 bdev(0x556c8bcf5c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:38:32.993 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:32.986+0000 7fd345e3f8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:38:35.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:38:35.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:38:35.275 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:38:35.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:38:35.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:38:35.470 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:38:35.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:38:35.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:38:35.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:38:35.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:38:35.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:38:35.495 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:35.490+0000 7f3a193a98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:35.496 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:35.490+0000 7f3a193a98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:35.497 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:35.494+0000 7f3a193a98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:35.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:38:35.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:38:35.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:38:35.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:38:35.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:38:35.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:35.620 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:38:35.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:38:35.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:35.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:38:35.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:36.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:36.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:36.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:38:36.773 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:38:36.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:36.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:38:36.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:36.979 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:36.974+0000 7f3a193a98c0 -1 Falling back to public interface 2026-02-17T05:38:37.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:37.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:37.927 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:38:37.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:38:37.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:37.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:38:38.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:39.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:39.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:39.078 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:38:39.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:38:39.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:39.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:38:39.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:39.942 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:39.937+0000 7f3a193a98c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:38:40.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:40.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:40.229 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:38:40.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:38:40.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:40.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:38:40.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:41.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:41.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:41.378 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:38:41.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:38:41.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:41.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:38:41.528 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2298646869,v1:127.0.0.1:6811/2298646869] [v2:127.0.0.1:6812/2298646869,v1:127.0.0.1:6813/2298646869] exists,up bbd22585-72ca-44e1-9bd4-763ee2a331f9 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:549: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:38:41.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:38:41.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:38:41.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:38:41.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:38:41.530 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:38:41.530 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:38:41.530 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:38:41.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:38:41.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:38:41.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:38:41.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=31b637bf-c174-43c4-9752-c356e641f6f3 2026-02-17T05:38:41.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 31b637bf-c174-43c4-9752-c356e641f6f3' 2026-02-17T05:38:41.534 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 31b637bf-c174-43c4-9752-c356e641f6f3 2026-02-17T05:38:41.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:38:41.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDh/pNp17OFIBAArVyC52Y0oaWHO/rEyYaGWQ== 2026-02-17T05:38:41.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDh/pNp17OFIBAArVyC52Y0oaWHO/rEyYaGWQ=="}' 2026-02-17T05:38:41.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 31b637bf-c174-43c4-9752-c356e641f6f3 -i td/osd-scrub-repair/2/new.json 2026-02-17T05:38:41.701 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:38:41.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:38:41.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDh/pNp17OFIBAArVyC52Y0oaWHO/rEyYaGWQ== --osd-uuid 31b637bf-c174-43c4-9752-c356e641f6f3 2026-02-17T05:38:41.727 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:41.721+0000 7fe3f35f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:41.729 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:41.721+0000 7fe3f35f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:41.730 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:41.725+0000 7fe3f35f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:41.730 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:41.725+0000 7fe3f35f18c0 -1 bdev(0x558b24749c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:38:41.730 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:41.725+0000 7fe3f35f18c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:38:44.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:38:44.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:38:44.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:38:44.033 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:38:44.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:38:44.228 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:38:44.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:38:44.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:38:44.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:38:44.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:38:44.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:38:44.254 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:44.249+0000 7ff088fef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:44.255 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:44.249+0000 7ff088fef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:44.256 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:44.249+0000 7ff088fef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:38:44.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:38:44.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:44.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:38:44.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:45.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:45.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:45.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:38:45.529 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:38:45.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:45.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:38:45.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:45.739 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:45.733+0000 7ff088fef8c0 -1 Falling back to public interface 2026-02-17T05:38:46.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:46.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:46.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:38:46.684 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:38:46.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:46.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:38:46.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:47.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:47.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:47.838 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:38:47.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:38:47.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:47.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:38:47.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:48.737 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:38:48.729+0000 7ff088fef8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:38:48.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:48.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:48.988 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:38:48.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:38:48.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:48.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:38:49.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:38:50.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:38:50.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:38:50.140 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:38:50.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:38:50.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:38:50.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:38:50.286 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2538106165,v1:127.0.0.1:6819/2538106165] [v2:127.0.0.1:6820/2538106165,v1:127.0.0.1:6821/2538106165] exists,up 31b637bf-c174-43c4-9752-c356e641f6f3 2026-02-17T05:38:50.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:38:50.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:38:50.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:38:50.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:552: TEST_auto_repair_bluestore_failed: create_pool testpool 1 1 2026-02-17T05:38:50.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T05:38:50.482 INFO:tasks.workunit.client.0.trial074.stderr:pool 'testpool' created 2026-02-17T05:38:50.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:38:51.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:553: TEST_auto_repair_bluestore_failed: ceph osd pool set testpool size 2 2026-02-17T05:38:51.644 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 size to 2 2026-02-17T05:38:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:554: TEST_auto_repair_bluestore_failed: wait_for_clean 2026-02-17T05:38:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:38:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:38:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:38:51.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:38:51.655 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:38:51.655 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:38:51.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:38:51.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:38:51.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:38:51.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:38:51.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:38:51.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:38:51.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:38:51.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:38:51.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:38:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:38:51.911 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:38:51.911 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:38:51.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:38:51.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:51.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:38:51.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T05:38:51.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T05:38:51.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T05:38:51.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:51.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:38:52.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T05:38:52.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T05:38:52.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T05:38:52.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:38:52.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:38:52.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T05:38:52.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T05:38:52.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542146' 2026-02-17T05:38:52.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:52.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T05:38:52.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:52.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:38:52.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T05:38:52.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:52.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T05:38:52.127 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:38:52.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T05:38:52.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:38:52.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-17T05:38:52.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:52.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T05:38:52.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:52.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:38:52.290 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T05:38:52.290 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:52.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T05:38:52.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T05:38:52.292 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T05:38:52.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:38:52.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672963 -lt 42949672964 2026-02-17T05:38:52.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:38:53.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:38:53.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:38:53.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672963 -lt 42949672964 2026-02-17T05:38:53.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:38:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:38:54.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:38:54.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T05:38:54.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:38:54.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T05:38:54.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:38:54.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:38:54.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T05:38:54.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:38:54.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T05:38:54.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T05:38:54.764 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542146 2026-02-17T05:38:54.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:38:54.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T05:38:54.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:38:54.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:54.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:55.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:38:55.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:55.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:55.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:55.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:55.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:55.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:55.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:55.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:55.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:55.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:55.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:55.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:55.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T05:38:55.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:38:55.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T05:38:55.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:55.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:55.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:55.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:55.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:55.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:55.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:38:55.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:38:56.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:38:56.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:38:56.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:38:56.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:56.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:38:56.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:56.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:56.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:56.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:56.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:56.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:56.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:56.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:56.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:56.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:56.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:56.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:56.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:56.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:38:56.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:38:56.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:38:56.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:38:56.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:38:56.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:38:56.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:38:56.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:38:56.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:38:56.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:38:56.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T05:38:56.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:56.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T05:38:57.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:57.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:57.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:57.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:57.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:57.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:57.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:57.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:57.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:38:57.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:57.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:57.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:38:57.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:38:57.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:38:57.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:38:57.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:38:57.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T05:38:57.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:38:57.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T05:38:58.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:38:58.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:38:58.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:38:58.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:38:58.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:38:58.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:38:58.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:38:58.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:38:58.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:38:58.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:38:58.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:38:58.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:38:58.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:38:58.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:38:58.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:38:58.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:557: TEST_auto_repair_bluestore_failed: local payload=ABCDEF 2026-02-17T05:38:58.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:558: TEST_auto_repair_bluestore_failed: echo ABCDEF 2026-02-17T05:38:58.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: seq 1 10 2026-02-17T05:38:58.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:58.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:58.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:59.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T05:38:59.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:38:59.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:566: TEST_auto_repair_bluestore_failed: get_not_primary testpool SOMETHING 2026-02-17T05:38:59.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:38:59.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:38:59.033 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:38:59.033 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:38:59.033 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:38:59.034 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:38:59.034 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:38:59.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:38:59.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:38:59.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:566: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj1 remove 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 remove 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:38:59.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 remove 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:39:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 remove 2026-02-17T05:39:01.295 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T05:39:01.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:01.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:39:01.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:39:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:39:01.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:39:01.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:39:01.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:39:01.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:39:01.835 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:39:01.835 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:39:01.835 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:39:01.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:39:01.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:39:01.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:39:01.859 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:01.853+0000 7f20a5b978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:01.860 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:01.853+0000 7f20a5b978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:01.861 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:01.857+0000 7f20a5b978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:39:01.988 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:39:01.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:01.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:02.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:03.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:03.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:03.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:39:03.147 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:03.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:03.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:03.175 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:03.169+0000 7f20a5b978c0 -1 Falling back to public interface 2026-02-17T05:39:03.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:04.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:04.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:04.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:39:04.296 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:39:04.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:04.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:04.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:05.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:05.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:05.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:39:05.450 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:39:05.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:05.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:05.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:06.163 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:06.157+0000 7f20a5b978c0 -1 osd.0 20 log_to_monitors true 2026-02-17T05:39:06.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:06.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:06.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:39:06.603 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:39:06.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:06.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:06.751 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 24 up_thru 0 down_at 21 last_clean_interval [5,20) [v2:127.0.0.1:6802/111581107,v1:127.0.0.1:6803/111581107] [v2:127.0.0.1:6804/111581107,v1:127.0.0.1:6805/111581107] exists,up 2125a123-d1d1-4e15-9ae1-b0ee620e04fa 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:39:06.752 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:39:06.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:39:06.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:39:06.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:39:06.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:39:06.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:39:06.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:39:06.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:39:06.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:39:06.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:39:07.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:39:07.006 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:07.006 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:39:07.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:39:07.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:07.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:39:07.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215106 2026-02-17T05:39:07.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215106 2026-02-17T05:39:07.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106' 2026-02-17T05:39:07.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:07.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:39:07.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672968 2026-02-17T05:39:07.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672968 2026-02-17T05:39:07.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-42949672968' 2026-02-17T05:39:07.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:07.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:39:07.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-17T05:39:07.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-17T05:39:07.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-42949672968 2-60129542150' 2026-02-17T05:39:07.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:07.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215106 2026-02-17T05:39:07.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:07.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:39:07.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215106 2026-02-17T05:39:07.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:07.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215106 2026-02-17T05:39:07.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215106' 2026-02-17T05:39:07.228 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 103079215106 2026-02-17T05:39:07.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:07.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 103079215106 2026-02-17T05:39:07.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:39:08.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:39:08.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:08.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215106 -lt 103079215106 2026-02-17T05:39:08.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:08.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672968 2026-02-17T05:39:08.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:08.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:39:08.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672968 2026-02-17T05:39:08.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:08.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672968 2026-02-17T05:39:08.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-17T05:39:08.545 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672968 2026-02-17T05:39:08.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:08.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-17T05:39:08.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:08.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-17T05:39:08.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:08.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:39:08.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-17T05:39:08.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:08.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-17T05:39:08.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-17T05:39:08.711 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542150 2026-02-17T05:39:08.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:39:08.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-17T05:39:08.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:39:08.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:08.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:09.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:39:09.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:39:09.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:39:09.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:39:09.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:39:09.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:39:09.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:39:09.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:39:09.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:39:09.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:39:09.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:09.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:09.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:39:09.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:39:09.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:39:09.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:568: TEST_auto_repair_bluestore_failed: get_not_primary testpool SOMETHING 2026-02-17T05:39:09.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:39:09.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:39:09.389 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:39:09.389 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:39:09.389 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:39:09.389 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:39:09.390 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:39:09.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:39:09.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:39:09.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:39:09.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:568: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj2 remove 2026-02-17T05:39:09.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:39:09.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:39:09.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:39:09.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj2 remove 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:39:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:39:11.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T05:39:11.888 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T05:39:12.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:12.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:39:12.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:39:12.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:39:12.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:39:12.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:39:12.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:39:12.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:39:12.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:39:12.428 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:39:12.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:12.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:39:12.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:39:12.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:39:12.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:39:12.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:39:12.446 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:12.444+0000 7f164e5138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:12.448 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:12.444+0000 7f164e5138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:12.449 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:12.444+0000 7f164e5138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:12.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:12.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:13.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:13.712+0000 7f164e5138c0 -1 Falling back to public interface 2026-02-17T05:39:13.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:13.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:13.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:39:13.733 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:13.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:13.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:13.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:14.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:14.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:14.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:39:14.882 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:39:14.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:14.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:15.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:16.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:16.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:16.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:39:16.033 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:39:16.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:16.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:16.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:16.718 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:16.716+0000 7f164e5138c0 -1 osd.0 25 log_to_monitors true 2026-02-17T05:39:17.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:17.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:17.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:39:17.181 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:39:17.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:17.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:17.331 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 29 up_thru 0 down_at 26 last_clean_interval [24,25) [v2:127.0.0.1:6802/3577901092,v1:127.0.0.1:6803/3577901092] [v2:127.0.0.1:6804/3577901092,v1:127.0.0.1:6805/3577901092] exists,up 2125a123-d1d1-4e15-9ae1-b0ee620e04fa 2026-02-17T05:39:17.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:39:17.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:39:17.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:39:17.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:39:17.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:39:17.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:39:17.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:39:17.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:39:17.333 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:39:17.333 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:39:17.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:39:17.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:39:17.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:39:17.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:39:17.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:39:17.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:39:17.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:39:17.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:39:17.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:39:17.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:39:17.580 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:17.580 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:39:17.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:39:17.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:17.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:39:17.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T05:39:17.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T05:39:17.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586' 2026-02-17T05:39:17.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:17.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:39:17.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672971 2026-02-17T05:39:17.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672971 2026-02-17T05:39:17.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-42949672971' 2026-02-17T05:39:17.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:17.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:39:17.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-17T05:39:17.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-17T05:39:17.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-42949672971 2-60129542153' 2026-02-17T05:39:17.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:17.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051586 2026-02-17T05:39:17.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:17.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:39:17.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051586 2026-02-17T05:39:17.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:17.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T05:39:17.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-17T05:39:17.795 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 124554051586 2026-02-17T05:39:17.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:17.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215106 -lt 124554051586 2026-02-17T05:39:17.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:39:18.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:39:18.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:19.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T05:39:19.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:19.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672971 2026-02-17T05:39:19.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:19.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:39:19.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672971 2026-02-17T05:39:19.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:19.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672971 2026-02-17T05:39:19.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-17T05:39:19.103 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672971 2026-02-17T05:39:19.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:19.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-17T05:39:19.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:19.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-17T05:39:19.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:19.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:39:19.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-17T05:39:19.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:19.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-17T05:39:19.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-17T05:39:19.257 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542153 2026-02-17T05:39:19.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:39:19.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-17T05:39:19.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:39:19.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:19.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:19.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:39:19.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:39:19.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:39:19.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:39:19.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:39:19.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:39:19.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:39:19.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:39:19.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:39:19.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:39:19.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:19.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:19.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:39:19.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:39:19.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:39:19.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:569: TEST_auto_repair_bluestore_failed: get_primary testpool SOMETHING 2026-02-17T05:39:19.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:39:19.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:39:19.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:39:19.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:39:20.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:569: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T05:39:20.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:39:20.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:39:20.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:39:20.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:39:21.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:39:21.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T05:39:21.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:39:21.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:39:21.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:39:21.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:39:21.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:39:21.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 rm-attr _ 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:39:22.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:39:22.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:39:22.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:39:22.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:39:22.605 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:39:22.605 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:39:22.605 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:39:22.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:39:22.606 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:39:22.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:22.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:39:22.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:39:22.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:39:22.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:39:22.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:39:22.626 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:22.624+0000 7fe5d089d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:22.627 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:22.624+0000 7fe5d089d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:22.628 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:22.624+0000 7fe5d089d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:22.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:22.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:22.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:23.892 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:23.888+0000 7fe5d089d8c0 -1 Falling back to public interface 2026-02-17T05:39:23.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:23.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:23.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:39:23.916 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:23.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:23.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:24.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:25.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:25.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:25.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:39:25.065 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:39:25.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:25.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:25.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:26.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:26.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:26.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:39:26.213 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:39:26.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:26.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:26.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:26.865 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:26.860+0000 7fe5d089d8c0 -1 osd.1 30 log_to_monitors true 2026-02-17T05:39:27.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:27.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:27.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:39:27.365 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:39:27.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:27.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:27.512 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 34 up_thru 34 down_at 31 last_clean_interval [10,30) [v2:127.0.0.1:6810/2541338812,v1:127.0.0.1:6811/2541338812] [v2:127.0.0.1:6812/2541338812,v1:127.0.0.1:6813/2541338812] exists,up bbd22585-72ca-44e1-9bd4-763ee2a331f9 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:39:27.513 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:39:27.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:39:27.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:39:27.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:39:27.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:39:27.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:39:27.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:39:27.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:39:27.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:39:27.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:39:27.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:39:27.780 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:27.780 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:39:27.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:39:27.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:27.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:39:27.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051589 2026-02-17T05:39:27.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051589 2026-02-17T05:39:27.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051589' 2026-02-17T05:39:27.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:27.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:39:27.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888066 2026-02-17T05:39:27.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888066 2026-02-17T05:39:27.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051589 1-146028888066' 2026-02-17T05:39:27.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:27.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:39:27.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542156 2026-02-17T05:39:27.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542156 2026-02-17T05:39:27.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051589 1-146028888066 2-60129542156' 2026-02-17T05:39:27.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:27.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051589 2026-02-17T05:39:27.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:27.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:39:27.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051589 2026-02-17T05:39:27.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:28.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051589 2026-02-17T05:39:28.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051589' 2026-02-17T05:39:28.000 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 124554051589 2026-02-17T05:39:28.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:28.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051589 -lt 124554051589 2026-02-17T05:39:28.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:28.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888066 2026-02-17T05:39:28.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:28.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:39:28.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888066 2026-02-17T05:39:28.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:28.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888066 2026-02-17T05:39:28.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888066' 2026-02-17T05:39:28.166 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 146028888066 2026-02-17T05:39:28.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:28.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888066 -lt 146028888066 2026-02-17T05:39:28.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:28.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542156 2026-02-17T05:39:28.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:28.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:39:28.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542156 2026-02-17T05:39:28.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:28.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542156 2026-02-17T05:39:28.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-17T05:39:28.322 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542156 2026-02-17T05:39:28.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:39:28.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-17T05:39:28.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:39:28.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:28.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:28.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:39:28.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:39:28.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:39:28.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:39:28.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:39:28.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:39:28.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:39:28.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:39:28.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:39:28.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:39:28.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:28.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:29.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:39:29.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:39:29.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:39:29.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:571: TEST_auto_repair_bluestore_failed: get_pg testpool obj1 2026-02-17T05:39:29.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T05:39:29.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T05:39:29.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T05:39:29.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:39:29.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:571: TEST_auto_repair_bluestore_failed: local pgid=1.0 2026-02-17T05:39:29.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:572: TEST_auto_repair_bluestore_failed: get_primary testpool obj1 2026-02-17T05:39:29.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:39:29.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T05:39:29.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T05:39:29.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:39:29.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:572: TEST_auto_repair_bluestore_failed: local primary=1 2026-02-17T05:39:29.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:573: TEST_auto_repair_bluestore_failed: get_last_scrub_stamp 1.0 2026-02-17T05:39:29.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:39:29.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:39:29.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:39:29.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:39:29.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:573: TEST_auto_repair_bluestore_failed: local last_scrub_stamp=2026-02-17T05:38:50.480300+0000 2026-02-17T05:39:29.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:574: TEST_auto_repair_bluestore_failed: ceph tell 1.0 schedule-deep-scrub 2026-02-17T05:39:29.520 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:39:29.521 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T05:39:29.521 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T05:39:29.521 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-03T05:37:49.519169+0000" 2026-02-17T05:39:29.521 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:577: TEST_auto_repair_bluestore_failed: wait_for_scrub 1.0 2026-02-17T05:38:50.480300+0000 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:38:50.480300+0000 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:39:29.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:39:29.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:39:29.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:39:29.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:38:50.480300+0000 '>' 2026-02-17T05:38:50.480300+0000 2026-02-17T05:39:29.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:39:30.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:39:30.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:39:30.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:39:30.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:39:30.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:39:30.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:39:30.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:39:30.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:38:50.480300+0000 '>' 2026-02-17T05:38:50.480300+0000 2026-02-17T05:39:30.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:39:31.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:39:31.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:39:31.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:39:31.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:39:31.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:39:31.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:39:31.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:39:31.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:38:50.480300+0000 '>' 2026-02-17T05:38:50.480300+0000 2026-02-17T05:39:31.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:39:32.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:39:32.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:39:32.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:39:32.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:39:32.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:39:32.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:39:32.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:39:33.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:39:30.855734+0000 '>' 2026-02-17T05:38:50.480300+0000 2026-02-17T05:39:33.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:39:33.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:578: TEST_auto_repair_bluestore_failed: wait_for_clean 2026-02-17T05:39:33.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:39:33.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:39:33.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:39:33.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:39:33.145 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:39:33.145 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:39:33.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:39:33.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:39:33.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:39:33.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:39:33.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:39:33.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:39:33.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:39:33.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:39:33.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:39:33.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:39:33.390 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:33.390 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:39:33.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:39:33.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:33.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:39:33.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051591 2026-02-17T05:39:33.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051591 2026-02-17T05:39:33.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051591' 2026-02-17T05:39:33.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:33.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:39:33.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888068 2026-02-17T05:39:33.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888068 2026-02-17T05:39:33.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051591 1-146028888068' 2026-02-17T05:39:33.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:33.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:39:33.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T05:39:33.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T05:39:33.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051591 1-146028888068 2-60129542158' 2026-02-17T05:39:33.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:33.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051591 2026-02-17T05:39:33.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:33.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:39:33.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051591 2026-02-17T05:39:33.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:33.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051591 2026-02-17T05:39:33.611 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 124554051591 2026-02-17T05:39:33.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051591' 2026-02-17T05:39:33.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:33.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051590 -lt 124554051591 2026-02-17T05:39:33.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:39:34.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:39:34.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:34.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051591 -lt 124554051591 2026-02-17T05:39:34.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:34.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888068 2026-02-17T05:39:34.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:34.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:39:34.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888068 2026-02-17T05:39:34.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:34.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888068 2026-02-17T05:39:34.921 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 146028888068 2026-02-17T05:39:34.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888068' 2026-02-17T05:39:34.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:35.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-17T05:39:35.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:35.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T05:39:35.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:35.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:39:35.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T05:39:35.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:35.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T05:39:35.084 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542158 2026-02-17T05:39:35.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T05:39:35.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:39:35.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-17T05:39:35.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:39:35.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:35.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:35.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:39:35.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:39:35.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:39:35.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:39:35.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:39:35.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:39:35.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:39:35.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:39:35.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:39:35.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:39:35.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:35.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:35.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:39:35.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:39:35.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:39:35.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:579: TEST_auto_repair_bluestore_failed: flush_pg_stats 2026-02-17T05:39:35.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:39:35.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:39:35.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:39:35.909 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:35.909 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:39:35.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:39:35.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:35.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:39:35.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051592 2026-02-17T05:39:35.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051592 2026-02-17T05:39:35.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051592' 2026-02-17T05:39:35.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:35.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:39:36.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888069 2026-02-17T05:39:36.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888069 2026-02-17T05:39:36.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051592 1-146028888069' 2026-02-17T05:39:36.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:36.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:39:36.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542160 2026-02-17T05:39:36.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542160 2026-02-17T05:39:36.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051592 1-146028888069 2-60129542160' 2026-02-17T05:39:36.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:36.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051592 2026-02-17T05:39:36.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:36.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:39:36.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051592 2026-02-17T05:39:36.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:36.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051592 2026-02-17T05:39:36.125 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 124554051592 2026-02-17T05:39:36.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051592' 2026-02-17T05:39:36.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:36.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051592 -lt 124554051592 2026-02-17T05:39:36.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:36.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888069 2026-02-17T05:39:36.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:36.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:39:36.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888069 2026-02-17T05:39:36.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:36.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888069 2026-02-17T05:39:36.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888069' 2026-02-17T05:39:36.285 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 146028888069 2026-02-17T05:39:36.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:36.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888069 2026-02-17T05:39:36.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:39:37.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:39:37.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:37.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888069 2026-02-17T05:39:37.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:39:38.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:39:38.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:38.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888070 -lt 146028888069 2026-02-17T05:39:38.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:38.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542160 2026-02-17T05:39:38.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:38.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:39:38.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542160 2026-02-17T05:39:38.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:38.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542160 2026-02-17T05:39:38.759 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542160 2026-02-17T05:39:38.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542160' 2026-02-17T05:39:38.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:39:38.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542160 -lt 60129542160 2026-02-17T05:39:38.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:580: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 15 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 20 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish Current 'required': 0 Planned 'req_scrub': 0 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 19 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] planned CHECK_REPAIR MUST_DEEP_SCRUB MUST_SCRUB] scrubber: scrub_finish before flags: CHECK_REPAIR REQ_SCRUB. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 19 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:39:38.910 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair [ 1.0: REQ_SCRUB ] ] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-17T05:39:38.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:581: TEST_auto_repair_bluestore_failed: grep -q 'scrub_finish.*still present after re-scrub' td/osd-scrub-repair/osd.1.log 2026-02-17T05:39:38.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:582: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T05:39:39.058 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:39:39.059 INFO:tasks.workunit.client.0.trial074.stdout:1.0 10 0 0 0 0 63 0 0 10 0 10 active+clean+inconsistent+failed_repair 2026-02-17T05:39:30.855787+0000 20'10 36:105 [1,0] 1 [1,0] 1 20'10 2026-02-17T05:39:30.855734+0000 20'10 2026-02-17T05:39:30.855734+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:39:30.855734+0000 10 0 2026-02-17T05:39:39.059 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:39:39.059 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:39:39.059 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:39:39.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:583: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T05:39:39.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:583: TEST_auto_repair_bluestore_failed: grep -q '^1.0.*+failed_repair' 2026-02-17T05:39:39.222 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:39:39.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:587: TEST_auto_repair_bluestore_failed: get_not_primary testpool obj1 2026-02-17T05:39:39.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:39:39.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T05:39:39.231 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool obj1 2026-02-17T05:39:39.231 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:39:39.231 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T05:39:39.231 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T05:39:39.232 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:39:39.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:39:39.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool obj1 2026-02-17T05:39:39.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:39:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:587: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj1 list-attrs 2026-02-17T05:39:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:39:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:39:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:39:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:39:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 list-attrs 2026-02-17T05:39:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:39:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:39:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:39:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:39:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 list-attrs 2026-02-17T05:39:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:39:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:39:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:39:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:39:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:39:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 list-attrs 2026-02-17T05:39:41.152 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:39:41.152 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:39:41.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:41.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:39:41.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:39:41.691 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:39:41.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:39:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:39:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:39:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:39:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:39:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:39:41.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:39:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:39:41.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:39:41.710 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:41.703+0000 7f776262b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:41.712 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:41.707+0000 7f776262b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:41.713 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:41.707+0000 7f776262b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:41.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:41.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:42.980 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:42.975+0000 7f776262b8c0 -1 Falling back to public interface 2026-02-17T05:39:42.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:42.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:42.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:39:42.995 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:42.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:42.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:43.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:44.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:44.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:44.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:39:44.146 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:39:44.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:44.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:44.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:45.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:45.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:45.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:39:45.297 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:39:45.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:45.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:45.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:45.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:45.931+0000 7f776262b8c0 -1 osd.0 36 log_to_monitors true 2026-02-17T05:39:46.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:46.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:46.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:39:46.447 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:39:46.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:46.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:39:46.593 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 40 up_thru 31 down_at 37 last_clean_interval [29,36) [v2:127.0.0.1:6802/29794083,v1:127.0.0.1:6803/29794083] [v2:127.0.0.1:6804/29794083,v1:127.0.0.1:6805/29794083] exists,up 2125a123-d1d1-4e15-9ae1-b0ee620e04fa 2026-02-17T05:39:46.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:39:46.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:39:46.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:39:46.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:39:46.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:39:46.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:39:46.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:39:46.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:39:46.595 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:39:46.595 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:39:46.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:39:46.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:39:46.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:39:46.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:39:46.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:39:46.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:39:46.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:39:46.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:39:46.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:39:46.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:39:46.848 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:46.848 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:39:46.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:39:46.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:46.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:39:46.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691842 2026-02-17T05:39:46.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691842 2026-02-17T05:39:46.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691842' 2026-02-17T05:39:46.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:46.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:39:46.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888073 2026-02-17T05:39:46.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888073 2026-02-17T05:39:46.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691842 1-146028888073' 2026-02-17T05:39:46.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:46.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:39:47.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542163 2026-02-17T05:39:47.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542163 2026-02-17T05:39:47.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691842 1-146028888073 2-60129542163' 2026-02-17T05:39:47.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:47.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-171798691842 2026-02-17T05:39:47.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:47.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:39:47.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-171798691842 2026-02-17T05:39:47.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:47.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691842 2026-02-17T05:39:47.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 171798691842' 2026-02-17T05:39:47.069 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 171798691842 2026-02-17T05:39:47.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:47.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051593 -lt 171798691842 2026-02-17T05:39:47.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:39:48.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:39:48.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:48.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-17T05:39:48.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:48.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888073 2026-02-17T05:39:48.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:48.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:39:48.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888073 2026-02-17T05:39:48.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:48.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888073 2026-02-17T05:39:48.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888073' 2026-02-17T05:39:48.390 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 146028888073 2026-02-17T05:39:48.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:48.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888073 -lt 146028888073 2026-02-17T05:39:48.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:48.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542163 2026-02-17T05:39:48.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:48.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:39:48.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542163 2026-02-17T05:39:48.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:48.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542163 2026-02-17T05:39:48.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542163' 2026-02-17T05:39:48.554 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542163 2026-02-17T05:39:48.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:39:48.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542163 -lt 60129542163 2026-02-17T05:39:48.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:39:48.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:48.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:48.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:39:48.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:39:48.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:39:48.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:39:48.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:39:48.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:39:48.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:39:48.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:39:49.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:39:49.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:39:49.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:49.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:49.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:39:49.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:39:49.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:39:49.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:588: TEST_auto_repair_bluestore_failed: rados --pool testpool get obj1 td/osd-scrub-repair/COPY 2026-02-17T05:39:49.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:589: TEST_auto_repair_bluestore_failed: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:39:49.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:590: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T05:39:49.259 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 15 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 20 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish Current 'required': 0 Planned 'req_scrub': 0 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 19 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] planned CHECK_REPAIR MUST_DEEP_SCRUB MUST_SCRUB] scrubber: scrub_finish before flags: CHECK_REPAIR REQ_SCRUB. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 19 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:39:49.260 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair [ 1.0: REQ_SCRUB ] ] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-17T05:39:49.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:593: TEST_auto_repair_bluestore_failed: get_primary testpool SOMETHING 2026-02-17T05:39:49.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:39:49.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:39:49.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:39:49.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:593: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 1 obj2 remove 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 remove 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:39:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 remove 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:39:50.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 remove 2026-02-17T05:39:51.372 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:104778fc:::obj2:head#, (61) No data available 2026-02-17T05:39:51.372 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T05:39:51.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:51.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:39:51.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:39:51.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:39:51.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:39:51.909 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:39:51.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:39:51.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:39:51.911 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:39:51.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:39:51.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:39:51.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:39:51.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:39:51.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:39:51.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:39:51.931 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:51.927+0000 7efeaf1c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:51.932 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:51.927+0000 7efeaf1c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:51.933 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:51.927+0000 7efeaf1c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:39:52.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:39:52.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:39:52.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:39:52.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:39:52.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:39:52.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:52.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:39:52.067 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:39:52.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:52.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:52.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:53.185 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:53.179+0000 7efeaf1c98c0 -1 Falling back to public interface 2026-02-17T05:39:53.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:53.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:53.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:39:53.218 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:53.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:53.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:53.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:54.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:54.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:54.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:39:54.363 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:39:54.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:54.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:54.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:55.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:55.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:55.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:39:55.518 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:39:55.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:55.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:55.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:39:56.162 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:39:56.155+0000 7efeaf1c98c0 -1 osd.1 41 log_to_monitors true 2026-02-17T05:39:56.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:39:56.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:39:56.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:39:56.667 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:39:56.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:39:56.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:39:56.820 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 42 last_clean_interval [34,41) [v2:127.0.0.1:6810/1503039136,v1:127.0.0.1:6811/1503039136] [v2:127.0.0.1:6812/1503039136,v1:127.0.0.1:6813/1503039136] exists,up bbd22585-72ca-44e1-9bd4-763ee2a331f9 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:39:56.821 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:39:56.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:39:56.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:39:56.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:39:56.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:39:56.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:39:56.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:39:56.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:39:56.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:39:56.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:39:57.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:39:57.087 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:39:57.087 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:39:57.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:39:57.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:57.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:39:57.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691845 2026-02-17T05:39:57.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691845 2026-02-17T05:39:57.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691845' 2026-02-17T05:39:57.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:57.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:39:57.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528322 2026-02-17T05:39:57.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528322 2026-02-17T05:39:57.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691845 1-193273528322' 2026-02-17T05:39:57.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:39:57.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:39:57.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542166 2026-02-17T05:39:57.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542166 2026-02-17T05:39:57.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691845 1-193273528322 2-60129542166' 2026-02-17T05:39:57.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:57.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-171798691845 2026-02-17T05:39:57.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:57.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:39:57.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-171798691845 2026-02-17T05:39:57.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:57.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691845 2026-02-17T05:39:57.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 171798691845' 2026-02-17T05:39:57.309 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 171798691845 2026-02-17T05:39:57.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:57.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691843 -lt 171798691845 2026-02-17T05:39:57.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:39:58.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:39:58.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:39:58.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691845 -lt 171798691845 2026-02-17T05:39:58.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:58.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-193273528322 2026-02-17T05:39:58.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:58.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:39:58.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-193273528322 2026-02-17T05:39:58.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:58.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528322 2026-02-17T05:39:58.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 193273528322' 2026-02-17T05:39:58.624 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 193273528322 2026-02-17T05:39:58.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:39:58.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528322 -lt 193273528322 2026-02-17T05:39:58.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:39:58.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542166 2026-02-17T05:39:58.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:39:58.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:39:58.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542166 2026-02-17T05:39:58.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:39:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542166 2026-02-17T05:39:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542166' 2026-02-17T05:39:58.781 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542166 2026-02-17T05:39:58.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:39:58.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542166 -lt 60129542166 2026-02-17T05:39:58.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:39:58.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:58.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:39:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:39:59.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:39:59.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:39:59.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:39:59.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:39:59.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:39:59.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:39:59.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:39:59.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:39:59.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:39:59.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:39:59.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:594: TEST_auto_repair_bluestore_failed: repair 1.0 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:39:59.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:39:59.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:39:30.855734+0000 2026-02-17T05:39:59.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T05:39:59.744 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T05:39:59.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T05:39:30.855734+0000 2026-02-17T05:39:59.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T05:39:59.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:39:30.855734+0000 2026-02-17T05:39:59.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:39:59.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:39:59.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:39:59.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:39:59.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:39:59.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:39:59.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:39:59.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:39:59.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:39:30.855734+0000 '>' 2026-02-17T05:39:30.855734+0000 2026-02-17T05:39:59.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:40:00.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:40:00.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:40:00.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:40:00.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:40:00.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:40:00.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:40:00.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:40:01.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:39:30.855734+0000 '>' 2026-02-17T05:39:30.855734+0000 2026-02-17T05:40:01.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:40:02.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:40:02.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:40:02.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:40:02.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:40:02.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:40:02.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:40:02.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:40:02.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:40:00.169991+0000 '>' 2026-02-17T05:39:30.855734+0000 2026-02-17T05:40:02.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:40:02.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:595: TEST_auto_repair_bluestore_failed: sleep 2 2026-02-17T05:40:04.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:597: TEST_auto_repair_bluestore_failed: flush_pg_stats 2026-02-17T05:40:04.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:40:04.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:40:04.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:40:04.365 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:40:04.365 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:40:04.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:40:04.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:04.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:40:04.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691847 2026-02-17T05:40:04.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691847 2026-02-17T05:40:04.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691847' 2026-02-17T05:40:04.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:04.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:40:04.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528324 2026-02-17T05:40:04.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528324 2026-02-17T05:40:04.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691847 1-193273528324' 2026-02-17T05:40:04.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:04.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:40:04.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542168 2026-02-17T05:40:04.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542168 2026-02-17T05:40:04.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691847 1-193273528324 2-60129542168' 2026-02-17T05:40:04.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:04.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-171798691847 2026-02-17T05:40:04.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:04.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:40:04.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-171798691847 2026-02-17T05:40:04.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:04.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691847 2026-02-17T05:40:04.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 171798691847' 2026-02-17T05:40:04.585 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 171798691847 2026-02-17T05:40:04.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:40:04.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691846 -lt 171798691847 2026-02-17T05:40:04.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:40:05.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:40:05.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:40:05.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691846 -lt 171798691847 2026-02-17T05:40:05.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:40:06.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:40:06.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:40:07.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691847 -lt 171798691847 2026-02-17T05:40:07.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:07.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-193273528324 2026-02-17T05:40:07.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:07.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:40:07.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-193273528324 2026-02-17T05:40:07.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:07.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528324 2026-02-17T05:40:07.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 193273528324' 2026-02-17T05:40:07.059 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 193273528324 2026-02-17T05:40:07.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:40:07.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528324 -lt 193273528324 2026-02-17T05:40:07.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542168 2026-02-17T05:40:07.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:07.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:40:07.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542168 2026-02-17T05:40:07.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:07.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542168 2026-02-17T05:40:07.222 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542168 2026-02-17T05:40:07.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542168' 2026-02-17T05:40:07.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:40:07.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542169 -lt 60129542168 2026-02-17T05:40:07.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:598: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T05:40:07.521 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:40:07.521 INFO:tasks.workunit.client.0.trial074.stdout:1.0 9 0 0 0 0 63 0 0 10 0 10 active+clean 2026-02-17T05:40:00.170074+0000 20'10 46:143 [1,0] 1 [1,0] 1 20'10 2026-02-17T05:40:00.169991+0000 20'10 2026-02-17T05:40:00.169991+0000 0 1 periodic scrub scheduled @ 2026-02-18T05:40:00.169991+0000 9 0 2026-02-17T05:40:07.522 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:40:07.522 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:40:07.522 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:40:07.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:599: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T05:40:07.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:599: TEST_auto_repair_bluestore_failed: grep -q -e '^1.0.* active+clean ' -e '^1.0.* active+clean+wait ' 2026-02-17T05:40:07.690 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:40:07.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:600: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T05:40:07.700 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 15 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 20 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish Current 'required': 0 Planned 'req_scrub': 0 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.848+0000 7fe5ae81b640 19 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] planned CHECK_REPAIR MUST_DEEP_SCRUB MUST_SCRUB] scrubber: scrub_finish before flags: CHECK_REPAIR REQ_SCRUB. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 19 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:39:30.852+0000 7fe5ae81b640 10 osd.1 pg_epoch: 36 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=34/35 n=10 ec=16/16 lis/c=34/34 les/c/f=35/36/0 sis=34) [1,0] r=0 lpr=34 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair [ 1.0: REQ_SCRUB ] ] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:40:00.167+0000 7efe9114f640 10 osd.1 pg_epoch: 46 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=45/46 n=10 ec=16/16 lis/c=45/45 les/c/f=46/46/0 sis=45) [1,0] r=0 lpr=45 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] MUST_REPAIR planned AUTO_REPAIR MUST_DEEP_SCRUB MUST_SCRUB planned REQ_SCRUB] scrubber: scrub_finish before flags: AUTO_REPAIR REQ_SCRUB. repair state: repair. deep_scrub_on_error: 0 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:40:00.167+0000 7efe9114f640 10 osd.1 pg_epoch: 46 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=45/46 n=10 ec=16/16 lis/c=45/45 les/c/f=46/46/0 sis=45) [1,0] r=0 lpr=45 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:40:00.167+0000 7efe9114f640 15 osd.1 pg_epoch: 46 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=45/46 n=9 ec=16/16 lis/c=45/45 les/c/f=46/46/0 sis=45) [1,0] r=0 lpr=45 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:40:00.167+0000 7efe9114f640 20 osd.1 pg_epoch: 46 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=45/46 n=9 ec=16/16 lis/c=45/45 les/c/f=46/46/0 sis=45) [1,0] r=0 lpr=45 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish All may be fixed 2026-02-17T05:40:07.701 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:40:00.167+0000 7efe9114f640 19 osd.1 pg_epoch: 46 pg[1.0( v 20'10 (0'0,20'10] local-lis/les=45/46 n=9 ec=16/16 lis/c=45/45 les/c/f=46/46/0 sis=45) [1,0] r=0 lpr=45 crt=20'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:40:07.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:40:08.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:40:08.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:40:08.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:40:08.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:40:08.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:40:08.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:40:08.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:40:08.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:40:08.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:40:08.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:40:08.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:40:08.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:40:08.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:40:08.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:40:08.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:40:08.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:40:08.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:40:08.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:40:08.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:40:08.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:40:08.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:40:08.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:40:08.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:40:08.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:40:08.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:40:08.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:40:08.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:40:08.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:40:08.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:40:08.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:40:08.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:40:08.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:40:08.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:40:08.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:40:08.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:40:08.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:40:08.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:40:08.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:40:08.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:40:08.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:40:08.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:40:08.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:40:08.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_failed_norecov td/osd-scrub-repair 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:604: TEST_auto_repair_bluestore_failed_norecov: local dir=td/osd-scrub-repair 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:605: TEST_auto_repair_bluestore_failed_norecov: local poolname=testpool 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:608: TEST_auto_repair_bluestore_failed_norecov: run_mon td/osd-scrub-repair a 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:40:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:40:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:40:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:40:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:40:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:40:08.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:40:08.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:40:08.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:40:08.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:40:08.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:08.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:40:08.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:40:08.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:40:08.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:40:08.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:40:08.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:40:08.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:40:08.110 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:40:08.110 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:40:08.111 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:40:08.111 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:40:08.111 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.111 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.111 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:40:08.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:40:08.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:40:08.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:40:08.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:40:08.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:40:08.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:40:08.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:40:08.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:40:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:609: TEST_auto_repair_bluestore_failed_norecov: run_mgr td/osd-scrub-repair x 2026-02-17T05:40:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:40:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:40:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:40:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:40:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:40:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:40:08.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:40:08.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:40:08.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:40:08.336 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:40:08.336 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.336 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:08.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:40:08.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:40:08.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:612: TEST_auto_repair_bluestore_failed_norecov: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:40:08.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: seq 0 2 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:614: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:40:08.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:40:08.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:40:08.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:40:08.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:40:08.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:40:08.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=52915c3a-cda8-4b04-aa73-38f1bf7acce4 2026-02-17T05:40:08.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 52915c3a-cda8-4b04-aa73-38f1bf7acce4' 2026-02-17T05:40:08.361 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 52915c3a-cda8-4b04-aa73-38f1bf7acce4 2026-02-17T05:40:08.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:40:08.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA4/5NpAvkCFhAAfR1s+7kkjOvsWXfBX20qzg== 2026-02-17T05:40:08.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA4/5NpAvkCFhAAfR1s+7kkjOvsWXfBX20qzg=="}' 2026-02-17T05:40:08.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 52915c3a-cda8-4b04-aa73-38f1bf7acce4 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:40:08.462 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:40:08.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:40:08.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQA4/5NpAvkCFhAAfR1s+7kkjOvsWXfBX20qzg== --osd-uuid 52915c3a-cda8-4b04-aa73-38f1bf7acce4 2026-02-17T05:40:08.487 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:08.483+0000 7f8003acb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:08.489 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:08.483+0000 7f8003acb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:08.490 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:08.483+0000 7f8003acb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:08.490 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:08.483+0000 7f8003acb8c0 -1 bdev(0x55819669bc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:40:08.490 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:08.483+0000 7f8003acb8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:40:11.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:40:11.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:40:11.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:40:11.333 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:40:11.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:40:11.526 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:40:11.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:40:11.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:11.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:40:11.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:40:11.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:40:11.552 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:11.547+0000 7f90f58468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:11.554 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:11.547+0000 7f90f58468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:11.555 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:11.551+0000 7f90f58468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:40:11.676 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:40:11.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:11.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:11.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:12.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:12.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:12.832 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:40:12.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:40:12.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:12.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:12.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:13.025 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:13.018+0000 7f90f58468c0 -1 Falling back to public interface 2026-02-17T05:40:13.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:13.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:13.985 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:40:13.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:40:13.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:13.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:14.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:15.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:15.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:15.140 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:40:15.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:40:15.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:15.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:15.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:15.979 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:15.974+0000 7f90f58468c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:40:16.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:16.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:16.298 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:40:16.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:40:16.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:16.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:16.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:17.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:17.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:17.452 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:40:17.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:40:17.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:17.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:17.605 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2207606446,v1:127.0.0.1:6803/2207606446] [v2:127.0.0.1:6804/2207606446,v1:127.0.0.1:6805/2207606446] exists,up 52915c3a-cda8-4b04-aa73-38f1bf7acce4 2026-02-17T05:40:17.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:40:17.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:614: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:40:17.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:40:17.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:40:17.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:40:17.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2deab56b-2a16-42f8-9264-9cba71d4b9bd 2026-02-17T05:40:17.610 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 2deab56b-2a16-42f8-9264-9cba71d4b9bd 2026-02-17T05:40:17.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 2deab56b-2a16-42f8-9264-9cba71d4b9bd' 2026-02-17T05:40:17.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:40:17.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBB/5Np9FLcJBAAH6JrDlfzzalyzhRrGZ0usA== 2026-02-17T05:40:17.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBB/5Np9FLcJBAAH6JrDlfzzalyzhRrGZ0usA=="}' 2026-02-17T05:40:17.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2deab56b-2a16-42f8-9264-9cba71d4b9bd -i td/osd-scrub-repair/1/new.json 2026-02-17T05:40:17.776 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:40:17.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:40:17.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBB/5Np9FLcJBAAH6JrDlfzzalyzhRrGZ0usA== --osd-uuid 2deab56b-2a16-42f8-9264-9cba71d4b9bd 2026-02-17T05:40:17.802 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:17.794+0000 7f398c3798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:17.804 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:17.798+0000 7f398c3798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:17.805 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:17.798+0000 7f398c3798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:17.805 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:17.798+0000 7f398c3798c0 -1 bdev(0x55b2a5345c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:40:17.806 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:17.798+0000 7f398c3798c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:40:20.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:40:20.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:40:20.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:40:20.106 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:40:20.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:40:20.297 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:40:20.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:40:20.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:20.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:40:20.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:40:20.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:40:20.322 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:20.314+0000 7f47a734f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:20.324 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:20.318+0000 7f47a734f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:20.325 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:20.318+0000 7f47a734f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:20.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:40:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:40:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:40:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:40:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:40:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:40:20.450 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:40:20.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:20.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:20.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:21.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:21.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:21.605 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:40:21.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:40:21.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:21.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:21.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:21.825 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:21.818+0000 7f47a734f8c0 -1 Falling back to public interface 2026-02-17T05:40:22.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:22.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:22.756 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:40:22.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:40:22.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:22.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:22.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:23.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:23.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:23.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:40:23.905 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:40:23.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:23.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:24.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:24.832 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:24.826+0000 7f47a734f8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:40:25.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:25.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:25.058 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:40:25.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:40:25.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:25.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:25.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:26.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:26.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:26.217 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:40:26.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:40:26.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:26.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:26.366 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/811105612,v1:127.0.0.1:6811/811105612] [v2:127.0.0.1:6812/811105612,v1:127.0.0.1:6813/811105612] exists,up 2deab56b-2a16-42f8-9264-9cba71d4b9bd 2026-02-17T05:40:26.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:40:26.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:40:26.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:614: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:40:26.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:40:26.372 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:40:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:40:26.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:40:26.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:40:26.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:40:26.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:40:26.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:40:26.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8feb84fd-75a9-46ff-bcbc-72d92798d186 2026-02-17T05:40:26.375 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 8feb84fd-75a9-46ff-bcbc-72d92798d186 2026-02-17T05:40:26.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 8feb84fd-75a9-46ff-bcbc-72d92798d186' 2026-02-17T05:40:26.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:40:26.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBK/5NpJi+bFhAAohxIw3HpFi27lMhuCj+3pQ== 2026-02-17T05:40:26.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBK/5NpJi+bFhAAohxIw3HpFi27lMhuCj+3pQ=="}' 2026-02-17T05:40:26.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8feb84fd-75a9-46ff-bcbc-72d92798d186 -i td/osd-scrub-repair/2/new.json 2026-02-17T05:40:26.538 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:40:26.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:40:26.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBK/5NpJi+bFhAAohxIw3HpFi27lMhuCj+3pQ== --osd-uuid 8feb84fd-75a9-46ff-bcbc-72d92798d186 2026-02-17T05:40:26.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:26.558+0000 7f3c4834f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:26.566 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:26.558+0000 7f3c4834f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:26.567 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:26.558+0000 7f3c4834f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:26.568 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:26.562+0000 7f3c4834f8c0 -1 bdev(0x56256a221c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:40:26.568 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:26.562+0000 7f3c4834f8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:40:28.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:40:28.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:40:28.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:40:28.853 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:40:28.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:40:29.044 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:40:29.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:40:29.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:29.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:40:29.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:40:29.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:40:29.068 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:29.062+0000 7f74e4f5e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:29.069 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:29.062+0000 7f74e4f5e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:29.071 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:29.062+0000 7f74e4f5e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:29.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:40:29.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:40:29.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:40:29.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:40:29.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:40:29.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:29.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:40:29.197 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:40:29.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:29.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:40:29.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:30.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:30.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:30.348 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:40:30.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:40:30.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:30.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:40:30.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:30.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:30.558+0000 7f74e4f5e8c0 -1 Falling back to public interface 2026-02-17T05:40:31.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:31.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:31.501 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:40:31.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:40:31.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:31.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:40:31.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:32.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:32.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:32.652 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:40:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:40:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:40:32.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:33.584 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:33.578+0000 7f74e4f5e8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:40:33.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:33.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:33.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:40:33.802 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:40:33.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:33.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:40:33.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:34.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:34.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:34.951 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:40:34.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:40:34.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:34.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:40:35.101 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2558947787,v1:127.0.0.1:6819/2558947787] [v2:127.0.0.1:6820/2558947787,v1:127.0.0.1:6821/2558947787] exists,up 8feb84fd-75a9-46ff-bcbc-72d92798d186 2026-02-17T05:40:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:40:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:40:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:40:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:617: TEST_auto_repair_bluestore_failed_norecov: create_pool testpool 1 1 2026-02-17T05:40:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T05:40:35.300 INFO:tasks.workunit.client.0.trial074.stderr:pool 'testpool' created 2026-02-17T05:40:35.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:40:36.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed_norecov: ceph osd pool set testpool size 2 2026-02-17T05:40:36.503 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 size to 2 2026-02-17T05:40:36.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:619: TEST_auto_repair_bluestore_failed_norecov: wait_for_clean 2026-02-17T05:40:36.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:40:36.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:40:36.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:40:36.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:40:36.518 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:40:36.518 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:40:36.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:40:36.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:40:36.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:40:36.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:40:36.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:40:36.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:40:36.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:40:36.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:40:36.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:40:36.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:40:36.785 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:40:36.785 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:40:36.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:40:36.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:36.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:40:36.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T05:40:36.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T05:40:36.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T05:40:36.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:36.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:40:36.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T05:40:36.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T05:40:36.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T05:40:36.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:36.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:40:37.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T05:40:37.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T05:40:37.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542146' 2026-02-17T05:40:37.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:37.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T05:40:37.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:37.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:40:37.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T05:40:37.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:37.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T05:40:37.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T05:40:37.006 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:40:37.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:40:37.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:40:37.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:40:38.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:40:38.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:40:38.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:40:38.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:40:39.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:40:39.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:40:39.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T05:40:39.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:39.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T05:40:39.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:39.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:40:39.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T05:40:39.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:39.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T05:40:39.467 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T05:40:39.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T05:40:39.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:40:39.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T05:40:39.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:39.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T05:40:39.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:39.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:40:39.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T05:40:39.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:39.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T05:40:39.631 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542146 2026-02-17T05:40:39.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T05:40:39.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:40:39.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T05:40:39.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:40:39.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:40:39.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:40:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:40:40.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:40:40.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:40:40.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:40:40.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:40:40.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:40:40.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:40:40.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:40:40.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:40:40.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:40:40.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:40:40.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:40:40.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:40:40.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:40:40.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:40:40.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:622: TEST_auto_repair_bluestore_failed_norecov: local payload=ABCDEF 2026-02-17T05:40:40.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:623: TEST_auto_repair_bluestore_failed_norecov: echo ABCDEF 2026-02-17T05:40:40.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: seq 1 10 2026-02-17T05:40:40.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T05:40:40.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T05:40:40.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:632: TEST_auto_repair_bluestore_failed_norecov: get_not_primary testpool SOMETHING 2026-02-17T05:40:40.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:40:40.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:40:40.540 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:40:40.541 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:40:40.541 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:40:40.541 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:40:40.541 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:40:40.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:40:40.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:40:40.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:632: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 0 obj1 remove 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 remove 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:40:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:40:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:40:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:40:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:40:40.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:40:40.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:40:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:40:40.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:40:40.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:40:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:40:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 remove 2026-02-17T05:40:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:40:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:40:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:40:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:40:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:40:42.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 remove 2026-02-17T05:40:42.793 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T05:40:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:40:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:40:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:40:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:40:43.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:40:43.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:40:43.333 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:40:43.333 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:43.333 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:43.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:43.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:43.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:40:43.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:40:43.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:40:43.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:40:43.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:40:43.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:40:43.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:40:43.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:40:43.357 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:43.350+0000 7f32d927f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:43.359 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:43.350+0000 7f32d927f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:43.359 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:43.350+0000 7f32d927f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:43.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:43.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:44.605 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:44.597+0000 7f32d927f8c0 -1 Falling back to public interface 2026-02-17T05:40:44.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:44.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:44.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:40:44.637 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:40:44.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:44.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:44.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:45.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:45.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:45.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:40:45.788 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:40:45.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:45.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:45.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:46.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:46.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:46.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:40:46.938 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:40:46.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:46.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:47.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:47.599 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:47.589+0000 7f32d927f8c0 -1 osd.0 20 log_to_monitors true 2026-02-17T05:40:48.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:48.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:48.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:40:48.090 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:40:48.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:48.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:40:48.246 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 24 up_thru 0 down_at 21 last_clean_interval [5,20) [v2:127.0.0.1:6802/955990340,v1:127.0.0.1:6803/955990340] [v2:127.0.0.1:6804/955990340,v1:127.0.0.1:6805/955990340] exists,up 52915c3a-cda8-4b04-aa73-38f1bf7acce4 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:40:48.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:40:48.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:40:48.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:40:48.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:40:48.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:40:48.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:40:48.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:40:48.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:40:48.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:40:48.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:40:48.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:40:48.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:40:48.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:40:48.510 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:40:48.510 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:40:48.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:40:48.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:48.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:40:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215106 2026-02-17T05:40:48.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215106 2026-02-17T05:40:48.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106' 2026-02-17T05:40:48.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:48.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:40:48.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672967 2026-02-17T05:40:48.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672967 2026-02-17T05:40:48.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-42949672967' 2026-02-17T05:40:48.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:48.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:40:48.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542149 2026-02-17T05:40:48.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542149 2026-02-17T05:40:48.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-42949672967 2-60129542149' 2026-02-17T05:40:48.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:48.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215106 2026-02-17T05:40:48.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:48.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:40:48.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215106 2026-02-17T05:40:48.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:48.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215106 2026-02-17T05:40:48.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215106' 2026-02-17T05:40:48.729 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 103079215106 2026-02-17T05:40:48.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:40:48.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215106 -lt 103079215106 2026-02-17T05:40:48.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:48.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672967 2026-02-17T05:40:48.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:48.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:40:48.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672967 2026-02-17T05:40:48.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:48.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672967 2026-02-17T05:40:48.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-17T05:40:48.889 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672967 2026-02-17T05:40:48.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:40:49.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672967 2026-02-17T05:40:49.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:40:50.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:40:50.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:40:50.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672967 2026-02-17T05:40:50.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:40:51.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:40:51.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:40:51.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-17T05:40:51.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:40:51.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542149 2026-02-17T05:40:51.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:40:51.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:40:51.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542149 2026-02-17T05:40:51.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:40:51.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542149 2026-02-17T05:40:51.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-17T05:40:51.351 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542149 2026-02-17T05:40:51.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:40:51.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542149 2026-02-17T05:40:51.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:40:51.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:40:51.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:40:51.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:40:51.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:40:51.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:40:51.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:40:51.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:40:51.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:40:51.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:40:51.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:40:51.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:40:51.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:40:51.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:40:51.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:40:52.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:40:52.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:40:52.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:40:52.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:633: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool SOMETHING 2026-02-17T05:40:52.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:40:52.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:40:52.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:40:52.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:40:52.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:633: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:40:52.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:40:53.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:40:53.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-17T05:40:53.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:40:53.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:40:53.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:40:53.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:40:53.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:40:53.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 rm-attr _ 2026-02-17T05:40:54.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:54.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:40:54.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:40:54.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:40:54.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:40:54.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:40:54.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:40:54.642 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:40:54.642 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:40:54.642 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:40:54.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:54.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:40:54.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:40:54.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:40:54.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:40:54.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:40:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:40:54.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:40:54.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:40:54.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:40:54.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:54.657+0000 7f1495b648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:54.667 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:54.657+0000 7f1495b648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:54.668 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:54.661+0000 7f1495b648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:40:54.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:40:54.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:54.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:55.886 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:55.877+0000 7f1495b648c0 -1 Falling back to public interface 2026-02-17T05:40:55.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:55.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:55.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:40:55.951 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:40:55.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:55.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:56.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:57.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:57.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:57.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:40:57.103 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:40:57.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:57.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:57.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:58.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:58.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:58.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:40:58.254 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:40:58.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:58.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:58.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:40:58.875 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:40:58.865+0000 7f1495b648c0 -1 osd.1 25 log_to_monitors true 2026-02-17T05:40:59.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:40:59.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:40:59.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:40:59.406 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:40:59.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:40:59.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:40:59.554 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 26 last_clean_interval [10,25) [v2:127.0.0.1:6810/2314198142,v1:127.0.0.1:6811/2314198142] [v2:127.0.0.1:6812/2314198142,v1:127.0.0.1:6813/2314198142] exists,up 2deab56b-2a16-42f8-9264-9cba71d4b9bd 2026-02-17T05:40:59.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:40:59.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:40:59.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:40:59.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:40:59.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:40:59.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:40:59.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:40:59.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:40:59.555 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:40:59.555 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:40:59.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:40:59.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:40:59.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:40:59.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:40:59.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:40:59.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:40:59.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:40:59.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:40:59.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:40:59.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:40:59.819 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:40:59.820 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:40:59.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:40:59.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:59.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:40:59.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215109 2026-02-17T05:40:59.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215109 2026-02-17T05:40:59.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215109' 2026-02-17T05:40:59.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:59.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:40:59.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T05:40:59.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T05:40:59.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215109 1-124554051586' 2026-02-17T05:40:59.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:40:59.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:41:00.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-17T05:41:00.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-17T05:41:00.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215109 1-124554051586 2-60129542153' 2026-02-17T05:41:00.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:00.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215109 2026-02-17T05:41:00.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:00.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:41:00.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215109 2026-02-17T05:41:00.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:00.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215109 2026-02-17T05:41:00.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215109' 2026-02-17T05:41:00.035 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 103079215109 2026-02-17T05:41:00.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:00.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215108 -lt 103079215109 2026-02-17T05:41:00.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:41:01.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:41:01.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:01.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215109 2026-02-17T05:41:01.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:01.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051586 2026-02-17T05:41:01.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:01.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:41:01.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051586 2026-02-17T05:41:01.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:01.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T05:41:01.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-17T05:41:01.348 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 124554051586 2026-02-17T05:41:01.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:41:01.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T05:41:01.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:01.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-17T05:41:01.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:01.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:41:01.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-17T05:41:01.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:01.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-17T05:41:01.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-17T05:41:01.516 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542153 2026-02-17T05:41:01.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:41:01.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-17T05:41:01.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:41:01.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:01.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:01.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:41:01.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:41:01.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:41:01.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:41:01.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:41:01.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:41:01.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:41:01.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:41:02.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:41:02.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:41:02.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:02.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:41:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:41:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:41:02.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:635: TEST_auto_repair_bluestore_failed_norecov: get_not_primary testpool SOMETHING 2026-02-17T05:41:02.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:41:02.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:41:02.180 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:41:02.180 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:41:02.180 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:41:02.180 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:41:02.180 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:41:02.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:41:02.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:41:02.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:41:02.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:635: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 0 obj2 remove 2026-02-17T05:41:02.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:41:02.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj2 remove 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:41:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:41:02.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:41:03.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:41:03.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T05:41:03.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:41:03.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:41:03.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:41:03.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:41:03.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:41:03.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T05:41:04.429 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T05:41:04.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:41:04.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:04.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:04.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:41:04.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:41:04.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:41:04.989 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:04.981+0000 7f2e51a028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:04.990 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:04.981+0000 7f2e51a028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:04.991 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:04.981+0000 7f2e51a028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:05.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:41:05.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:41:05.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:41:05.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:41:05.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:41:05.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:05.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:41:05.123 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:41:05.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:05.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:05.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:06.234 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:06.225+0000 7f2e51a028c0 -1 Falling back to public interface 2026-02-17T05:41:06.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:06.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:06.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:41:06.274 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:41:06.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:06.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:06.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:07.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:07.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:07.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:41:07.421 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:41:07.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:07.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:07.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:08.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:08.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:08.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:41:08.570 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:41:08.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:08.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:08.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:09.203 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:09.193+0000 7f2e51a028c0 -1 osd.0 31 log_to_monitors true 2026-02-17T05:41:09.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:09.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:09.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:41:09.721 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:41:09.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:09.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:09.868 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 35 up_thru 26 down_at 32 last_clean_interval [24,31) [v2:127.0.0.1:6802/292320952,v1:127.0.0.1:6803/292320952] [v2:127.0.0.1:6804/292320952,v1:127.0.0.1:6805/292320952] exists,up 52915c3a-cda8-4b04-aa73-38f1bf7acce4 2026-02-17T05:41:09.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:41:09.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:41:09.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:41:09.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:41:09.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:41:09.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:41:09.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:41:09.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:41:09.870 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:41:09.870 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:41:09.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:41:09.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:41:09.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:41:09.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:41:09.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:41:09.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:41:09.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:41:09.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:41:09.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:41:10.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:41:10.133 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:41:10.133 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:41:10.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:41:10.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:10.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:41:10.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855362 2026-02-17T05:41:10.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855362 2026-02-17T05:41:10.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362' 2026-02-17T05:41:10.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:10.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:41:10.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051589 2026-02-17T05:41:10.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051589 2026-02-17T05:41:10.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-124554051589' 2026-02-17T05:41:10.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:10.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:41:10.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542156 2026-02-17T05:41:10.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542156 2026-02-17T05:41:10.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-124554051589 2-60129542156' 2026-02-17T05:41:10.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:10.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855362 2026-02-17T05:41:10.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:10.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:41:10.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855362 2026-02-17T05:41:10.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:10.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855362 2026-02-17T05:41:10.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855362' 2026-02-17T05:41:10.350 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 150323855362 2026-02-17T05:41:10.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:10.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 150323855362 2026-02-17T05:41:10.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:41:11.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:41:11.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:11.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855362 2026-02-17T05:41:11.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:11.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051589 2026-02-17T05:41:11.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:11.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:41:11.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051589 2026-02-17T05:41:11.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:11.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051589 2026-02-17T05:41:11.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051589' 2026-02-17T05:41:11.668 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 124554051589 2026-02-17T05:41:11.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:41:11.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051589 -lt 124554051589 2026-02-17T05:41:11.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:11.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542156 2026-02-17T05:41:11.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:11.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:41:11.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542156 2026-02-17T05:41:11.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:11.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542156 2026-02-17T05:41:11.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-17T05:41:11.835 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542156 2026-02-17T05:41:11.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:41:11.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-17T05:41:11.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:41:11.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:11.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:12.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:41:12.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:41:12.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:41:12.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:41:12.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:41:12.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:41:12.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:41:12.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:41:12.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:41:12.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:41:12.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:12.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:12.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:41:12.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:41:12.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:41:12.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:636: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool SOMETHING 2026-02-17T05:41:12.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:41:12.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:41:12.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:41:12.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:41:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:636: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T05:41:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:41:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:41:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:41:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:41:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T05:41:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:41:12.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:41:13.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:41:13.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T05:41:13.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:41:13.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:41:13.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:41:13.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:41:13.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:41:13.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 rm-attr _ 2026-02-17T05:41:14.646 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:ff7b1f36:::obj1:head#, (61) No data available 2026-02-17T05:41:15.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:41:15.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:41:15.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:41:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:41:15.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:41:15.188 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:41:15.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:41:15.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:41:15.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:41:15.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:41:15.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:41:15.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:41:15.205 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:15.200+0000 7fa2db79d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:15.207 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:15.200+0000 7fa2db79d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:15.208 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:15.200+0000 7fa2db79d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:15.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:15.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:15.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:16.475 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:16.468+0000 7fa2db79d8c0 -1 Falling back to public interface 2026-02-17T05:41:16.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:16.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:16.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:41:16.492 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:41:16.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:16.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:16.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:17.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:17.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:17.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:41:17.643 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:41:17.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:17.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:17.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:18.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:18.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:18.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:41:18.796 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:41:18.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:18.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:18.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:19.469 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:19.460+0000 7fa2db79d8c0 -1 osd.1 36 log_to_monitors true 2026-02-17T05:41:19.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:19.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:19.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:41:19.948 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:41:19.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:19.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [29,36) [v2:127.0.0.1:6810/3973158488,v1:127.0.0.1:6811/3973158488] [v2:127.0.0.1:6812/3973158488,v1:127.0.0.1:6813/3973158488] exists,up 2deab56b-2a16-42f8-9264-9cba71d4b9bd 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:41:20.100 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:41:20.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:41:20.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:41:20.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:41:20.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:41:20.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:41:20.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:41:20.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:41:20.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:41:20.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:41:20.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:41:20.361 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:41:20.361 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:41:20.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:41:20.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:20.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:41:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855365 2026-02-17T05:41:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855365 2026-02-17T05:41:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365' 2026-02-17T05:41:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:20.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:41:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691842 2026-02-17T05:41:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691842 2026-02-17T05:41:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365 1-171798691842' 2026-02-17T05:41:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:20.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:41:20.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542159 2026-02-17T05:41:20.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542159 2026-02-17T05:41:20.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365 1-171798691842 2-60129542159' 2026-02-17T05:41:20.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:20.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855365 2026-02-17T05:41:20.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:20.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:41:20.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855365 2026-02-17T05:41:20.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:20.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855365 2026-02-17T05:41:20.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855365' 2026-02-17T05:41:20.582 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 150323855365 2026-02-17T05:41:20.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:20.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-17T05:41:20.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:20.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691842 2026-02-17T05:41:20.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:20.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:41:20.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691842 2026-02-17T05:41:20.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:20.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691842 2026-02-17T05:41:20.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691842' 2026-02-17T05:41:20.733 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 171798691842 2026-02-17T05:41:20.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:41:20.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-17T05:41:20.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:20.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542159 2026-02-17T05:41:20.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:20.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:41:20.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542159 2026-02-17T05:41:20.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:20.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542159 2026-02-17T05:41:20.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542159' 2026-02-17T05:41:20.900 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542159 2026-02-17T05:41:20.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:41:21.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542159 -lt 60129542159 2026-02-17T05:41:21.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:41:21.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:21.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:21.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:41:21.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:41:21.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:41:21.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:41:21.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:41:21.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:41:21.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:41:21.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:41:21.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:41:21.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:41:21.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:21.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:21.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:41:21.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:41:21.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:41:21.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:637: TEST_auto_repair_bluestore_failed_norecov: ceph tell 'osd.*' config set osd_scrub_auto_repair true 2026-02-17T05:41:21.631 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T05:41:21.631 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T05:41:21.631 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:41:21.637 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T05:41:21.637 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T05:41:21.637 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:41:21.643 INFO:tasks.workunit.client.0.trial074.stdout:osd.2: { 2026-02-17T05:41:21.643 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-17T05:41:21.643 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:41:21.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:639: TEST_auto_repair_bluestore_failed_norecov: get_pg testpool obj1 2026-02-17T05:41:21.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T05:41:21.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T05:41:21.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T05:41:21.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:41:21.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:639: TEST_auto_repair_bluestore_failed_norecov: local pgid=1.0 2026-02-17T05:41:21.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:640: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool obj1 2026-02-17T05:41:21.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:41:21.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T05:41:21.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T05:41:21.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:41:21.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:640: TEST_auto_repair_bluestore_failed_norecov: local primary=1 2026-02-17T05:41:21.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:641: TEST_auto_repair_bluestore_failed_norecov: get_last_scrub_stamp 1.0 2026-02-17T05:41:21.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:41:21.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:41:21.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:41:21.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:41:22.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:641: TEST_auto_repair_bluestore_failed_norecov: local last_scrub_stamp=2026-02-17T05:40:35.296613+0000 2026-02-17T05:41:22.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:642: TEST_auto_repair_bluestore_failed_norecov: ceph tell 1.0 schedule-deep-scrub 2026-02-17T05:41:22.171 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:41:22.171 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T05:41:22.171 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T05:41:22.171 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-03T05:39:42.167177+0000" 2026-02-17T05:41:22.172 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:41:22.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:645: TEST_auto_repair_bluestore_failed_norecov: wait_for_scrub 1.0 2026-02-17T05:40:35.296613+0000 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:40:35.296613+0000 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:41:22.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:40:35.296613+0000 '>' 2026-02-17T05:40:35.296613+0000 2026-02-17T05:41:22.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:41:23.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:41:23.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:41:23.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:41:23.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:41:23.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:41:23.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:41:23.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:41:23.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:40:35.296613+0000 '>' 2026-02-17T05:40:35.296613+0000 2026-02-17T05:41:23.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:41:24.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:41:24.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:41:24.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:41:24.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:41:24.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:41:24.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:41:24.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:41:24.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:41:23.419564+0000 '>' 2026-02-17T05:40:35.296613+0000 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:646: TEST_auto_repair_bluestore_failed_norecov: wait_for_clean 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:41:24.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:41:24.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:41:24.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:41:24.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:41:24.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:41:24.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:41:24.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:41:24.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:41:24.891 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:41:24.891 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:41:24.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:41:24.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:24.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:41:24.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855367 2026-02-17T05:41:24.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855367 2026-02-17T05:41:24.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855367' 2026-02-17T05:41:24.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:24.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:41:25.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691844 2026-02-17T05:41:25.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691844 2026-02-17T05:41:25.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855367 1-171798691844' 2026-02-17T05:41:25.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:25.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:41:25.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542161 2026-02-17T05:41:25.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542161 2026-02-17T05:41:25.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855367 1-171798691844 2-60129542161' 2026-02-17T05:41:25.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:25.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855367 2026-02-17T05:41:25.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:25.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:41:25.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855367 2026-02-17T05:41:25.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:25.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855367 2026-02-17T05:41:25.111 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 150323855367 2026-02-17T05:41:25.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855367' 2026-02-17T05:41:25.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:25.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855366 -lt 150323855367 2026-02-17T05:41:25.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:41:26.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:41:26.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:26.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855366 -lt 150323855367 2026-02-17T05:41:26.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:41:27.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:41:27.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:27.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855367 -lt 150323855367 2026-02-17T05:41:27.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:27.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691844 2026-02-17T05:41:27.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:27.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:41:27.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691844 2026-02-17T05:41:27.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:27.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691844 2026-02-17T05:41:27.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691844' 2026-02-17T05:41:27.584 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 171798691844 2026-02-17T05:41:27.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:41:27.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691844 -lt 171798691844 2026-02-17T05:41:27.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:27.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542161 2026-02-17T05:41:27.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:27.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:41:27.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542161 2026-02-17T05:41:27.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:27.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542161 2026-02-17T05:41:27.748 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542161 2026-02-17T05:41:27.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542161' 2026-02-17T05:41:27.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:41:27.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542161 -lt 60129542161 2026-02-17T05:41:27.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:41:27.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:27.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:28.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:41:28.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:41:28.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:41:28.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:41:28.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:41:28.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:41:28.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:41:28.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:41:28.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:41:28.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:41:28.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:41:28.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:41:28.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:41:28.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:41:28.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:41:28.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:647: TEST_auto_repair_bluestore_failed_norecov: flush_pg_stats 2026-02-17T05:41:28.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:41:28.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:41:28.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:41:28.570 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:41:28.570 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:41:28.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:41:28.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:28.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:41:28.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855368 2026-02-17T05:41:28.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855368 2026-02-17T05:41:28.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855368' 2026-02-17T05:41:28.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:28.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:41:28.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691845 2026-02-17T05:41:28.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691845 2026-02-17T05:41:28.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855368 1-171798691845' 2026-02-17T05:41:28.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:41:28.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:41:28.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542162 2026-02-17T05:41:28.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542162 2026-02-17T05:41:28.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855368 1-171798691845 2-60129542162' 2026-02-17T05:41:28.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:28.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855368 2026-02-17T05:41:28.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:28.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:41:28.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855368 2026-02-17T05:41:28.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:28.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855368 2026-02-17T05:41:28.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855368' 2026-02-17T05:41:28.793 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 150323855368 2026-02-17T05:41:28.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:28.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855367 -lt 150323855368 2026-02-17T05:41:28.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:41:29.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:41:29.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:30.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855367 -lt 150323855368 2026-02-17T05:41:30.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:41:31.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:41:31.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:41:31.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855369 -lt 150323855368 2026-02-17T05:41:31.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:31.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691845 2026-02-17T05:41:31.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:31.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:41:31.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691845 2026-02-17T05:41:31.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:31.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691845 2026-02-17T05:41:31.258 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 171798691845 2026-02-17T05:41:31.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691845' 2026-02-17T05:41:31.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:41:31.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691846 -lt 171798691845 2026-02-17T05:41:31.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:41:31.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542162 2026-02-17T05:41:31.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:41:31.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:41:31.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542162 2026-02-17T05:41:31.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:41:31.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542162 2026-02-17T05:41:31.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542162' 2026-02-17T05:41:31.411 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542162 2026-02-17T05:41:31.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:41:31.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542163 -lt 60129542162 2026-02-17T05:41:31.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:648: TEST_auto_repair_bluestore_failed_norecov: grep -q 'scrub_finish.*present with no repair possible' td/osd-scrub-repair/osd.1.log 2026-02-17T05:41:31.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:649: TEST_auto_repair_bluestore_failed_norecov: ceph pg dump pgs 2026-02-17T05:41:31.715 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:41:31.715 INFO:tasks.workunit.client.0.trial074.stdout:1.0 10 0 0 0 0 56 0 0 10 0 10 active+clean+inconsistent+failed_repair 2026-02-17T05:41:23.419637+0000 20'10 41:101 [1,0] 1 [1,0] 1 20'10 2026-02-17T05:41:23.419564+0000 20'10 2026-02-17T05:41:23.419564+0000 0 1 periodic scrub scheduled @ 2026-02-18T05:41:23.419564+0000 10 0 2026-02-17T05:41:31.715 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:41:31.715 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:41:31.716 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:41:31.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:650: TEST_auto_repair_bluestore_failed_norecov: ceph pg dump pgs 2026-02-17T05:41:31.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:650: TEST_auto_repair_bluestore_failed_norecov: grep -q '^1.0.*+failed_repair' 2026-02-17T05:41:31.879 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:41:31.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:41:31.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:41:31.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:41:31.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:41:31.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:41:31.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:41:31.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:41:31.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:41:31.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:41:31.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:41:31.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:41:31.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:41:31.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:41:32.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:41:32.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:41:32.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:41:32.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:32.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:41:32.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:41:32.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:32.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:41:32.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:41:32.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:41:32.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:41:32.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:41:32.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:41:32.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:41:32.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:41:32.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:41:32.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:41:32.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:41:32.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:41:32.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:41:32.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:41:32.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:41:32.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:41:32.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:32.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:41:32.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:41:32.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:41:32.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:41:32.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:41:32.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:41:32.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:41:32.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:41:32.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:41:32.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:41:32.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:41:32.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:41:32.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:41:32.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:41:32.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:41:32.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:41:32.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:41:32.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_scrub td/osd-scrub-repair 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:483: TEST_auto_repair_bluestore_scrub: local dir=td/osd-scrub-repair 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:484: TEST_auto_repair_bluestore_scrub: local poolname=testpool 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:487: TEST_auto_repair_bluestore_scrub: run_mon td/osd-scrub-repair a 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:41:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:41:32.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:41:32.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:41:32.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:41:32.061 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:41:32.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:32.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:41:32.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:41:32.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:41:32.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:41:32.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:41:32.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:41:32.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:41:32.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:41:32.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:41:32.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:41:32.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:41:32.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:41:32.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:41:32.147 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:41:32.147 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:41:32.147 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:41:32.147 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:41:32.148 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.148 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.148 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:41:32.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:41:32.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:41:32.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:488: TEST_auto_repair_bluestore_scrub: run_mgr td/osd-scrub-repair x 2026-02-17T05:41:32.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:41:32.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:41:32.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:41:32.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:41:32.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:41:32.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:41:32.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:41:32.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:41:32.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:41:32.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:41:32.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:32.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:41:32.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:41:32.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:492: TEST_auto_repair_bluestore_scrub: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T05:41:32.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: seq 0 2 2026-02-17T05:41:32.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-17T05:41:32.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:494: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:41:32.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:32.316 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T05:41:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:41:32.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:41:32.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a1a4aa42-3afc-41e8-a4f0-e334872d22c3 2026-02-17T05:41:32.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a1a4aa42-3afc-41e8-a4f0-e334872d22c3' 2026-02-17T05:41:32.319 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 a1a4aa42-3afc-41e8-a4f0-e334872d22c3 2026-02-17T05:41:32.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:41:32.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCM/5Npp+VoExAAammLiag1uyamvHdI4e1coQ== 2026-02-17T05:41:32.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCM/5Npp+VoExAAammLiag1uyamvHdI4e1coQ=="}' 2026-02-17T05:41:32.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a1a4aa42-3afc-41e8-a4f0-e334872d22c3 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:41:32.420 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:41:32.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:41:32.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQCM/5Npp+VoExAAammLiag1uyamvHdI4e1coQ== --osd-uuid a1a4aa42-3afc-41e8-a4f0-e334872d22c3 2026-02-17T05:41:32.446 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:32.440+0000 7f3b659938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:32.448 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:32.440+0000 7f3b659938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:32.449 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:32.444+0000 7f3b659938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:32.450 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:32.444+0000 7f3b659938c0 -1 bdev(0x556a1246bc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:41:32.450 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:32.444+0000 7f3b659938c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:41:35.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:41:35.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:41:35.248 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:41:35.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:41:35.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:41:35.447 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:41:35.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:41:35.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:41:35.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:41:35.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:41:35.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:41:35.464 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:35.456+0000 7fc2a7c408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:35.465 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:35.460+0000 7fc2a7c408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:35.466 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:35.460+0000 7fc2a7c408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:35.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:41:35.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:41:35.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:41:35.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:41:35.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:41:35.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:35.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:41:35.598 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:41:35.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:35.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:35.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:36.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:36.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:36.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:41:36.749 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:41:36.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:36.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:36.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:36.955 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:36.948+0000 7fc2a7c408c0 -1 Falling back to public interface 2026-02-17T05:41:37.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:37.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:37.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:41:37.901 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:41:37.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:37.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:38.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:39.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:39.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:39.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:41:39.052 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:41:39.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:39.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:39.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:39.940 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:39.932+0000 7fc2a7c408c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:41:40.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:40.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:40.205 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:41:40.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:41:40.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:40.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:40.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:41.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:41.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:41.357 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:41:41.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:41:41.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:41.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:41:41.506 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3135301907,v1:127.0.0.1:6803/3135301907] [v2:127.0.0.1:6804/3135301907,v1:127.0.0.1:6805/3135301907] exists,up a1a4aa42-3afc-41e8-a4f0-e334872d22c3 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:494: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:41:41.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:41.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:41.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:41:41.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:41:41.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8dbba8c0-c4ab-463c-b2c2-896596e69033 2026-02-17T05:41:41.512 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 8dbba8c0-c4ab-463c-b2c2-896596e69033 2026-02-17T05:41:41.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 8dbba8c0-c4ab-463c-b2c2-896596e69033' 2026-02-17T05:41:41.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:41:41.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCV/5NpTmbnHhAA7pk4Px73j6sHS7+8zFRd8Q== 2026-02-17T05:41:41.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCV/5NpTmbnHhAA7pk4Px73j6sHS7+8zFRd8Q=="}' 2026-02-17T05:41:41.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8dbba8c0-c4ab-463c-b2c2-896596e69033 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:41:41.674 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:41:41.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:41:41.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQCV/5NpTmbnHhAA7pk4Px73j6sHS7+8zFRd8Q== --osd-uuid 8dbba8c0-c4ab-463c-b2c2-896596e69033 2026-02-17T05:41:41.699 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:41.692+0000 7f42e63738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:41.701 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:41.692+0000 7f42e63738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:41.702 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:41.696+0000 7f42e63738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:41.702 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:41.696+0000 7f42e63738c0 -1 bdev(0x55db0bdf7c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:41:41.702 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:41.696+0000 7f42e63738c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:41:43.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:41:43.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:41:43.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:41:43.979 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:41:43.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:41:44.167 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:41:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:41:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:41:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:41:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:41:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:41:44.193 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:44.184+0000 7f22545788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:44.195 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:44.188+0000 7f22545788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:44.196 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:44.188+0000 7f22545788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:41:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:41:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:41:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:41:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:41:44.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:44.322 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:41:44.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:41:44.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:44.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:44.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:45.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:45.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:45.475 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:41:45.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:41:45.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:45.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:45.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:45.703 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:45.696+0000 7f22545788c0 -1 Falling back to public interface 2026-02-17T05:41:46.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:46.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:46.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:41:46.626 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:41:46.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:46.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:46.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:47.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:47.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:47.776 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:41:47.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:41:47.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:47.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:47.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:48.701 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:48.691+0000 7f22545788c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:41:48.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:48.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:48.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:41:48.927 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:41:48.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:48.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:49.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:50.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:50.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:50.078 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:41:50.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:41:50.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:50.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:41:50.227 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2147172278,v1:127.0.0.1:6811/2147172278] [v2:127.0.0.1:6812/2147172278,v1:127.0.0.1:6813/2147172278] exists,up 8dbba8c0-c4ab-463c-b2c2-896596e69033 2026-02-17T05:41:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:41:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:41:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:41:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-17T05:41:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:494: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:41:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:41:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:41:50.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:41:50.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:41:50.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=54bbdddc-ab8f-4282-83d3-319ebb101806 2026-02-17T05:41:50.235 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 54bbdddc-ab8f-4282-83d3-319ebb101806 2026-02-17T05:41:50.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 54bbdddc-ab8f-4282-83d3-319ebb101806' 2026-02-17T05:41:50.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:41:50.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCe/5Np1ZooDhAALzDWL9URFujJcbLEP+30ZA== 2026-02-17T05:41:50.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCe/5Np1ZooDhAALzDWL9URFujJcbLEP+30ZA=="}' 2026-02-17T05:41:50.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 54bbdddc-ab8f-4282-83d3-319ebb101806 -i td/osd-scrub-repair/2/new.json 2026-02-17T05:41:50.389 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:41:50.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:41:50.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQCe/5Np1ZooDhAALzDWL9URFujJcbLEP+30ZA== --osd-uuid 54bbdddc-ab8f-4282-83d3-319ebb101806 2026-02-17T05:41:50.416 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:50.407+0000 7fc919d228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:50.417 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:50.411+0000 7fc919d228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:50.418 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:50.411+0000 7fc919d228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:50.419 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:50.411+0000 7fc919d228c0 -1 bdev(0x55d2d4209c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:41:50.419 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:50.411+0000 7fc919d228c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:41:52.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:41:52.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:41:52.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:41:52.672 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:41:52.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:41:52.867 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:41:52.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:41:52.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:41:52.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:41:52.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:41:52.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:41:52.893 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:52.883+0000 7ff6afe5a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:52.895 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:52.887+0000 7ff6afe5a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:52.896 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:52.887+0000 7ff6afe5a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:41:53.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:41:53.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:41:53.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:41:53.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:41:53.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:41:53.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:53.019 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:41:53.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:41:53.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:53.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:41:53.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:54.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:54.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:54.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:41:54.173 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:41:54.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:54.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:41:54.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:54.379 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:54.371+0000 7ff6afe5a8c0 -1 Falling back to public interface 2026-02-17T05:41:55.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:55.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:55.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:41:55.324 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:41:55.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:55.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:41:55.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:56.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:56.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:56.477 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:41:56.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:41:56.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:56.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:41:56.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:57.415 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:41:57.407+0000 7ff6afe5a8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:41:57.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:57.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:57.629 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:41:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:41:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:41:57.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:41:58.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:41:58.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:41:58.785 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:41:58.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:41:58.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:41:58.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:41:58.932 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2951421690,v1:127.0.0.1:6819/2951421690] [v2:127.0.0.1:6820/2951421690,v1:127.0.0.1:6821/2951421690] exists,up 54bbdddc-ab8f-4282-83d3-319ebb101806 2026-02-17T05:41:58.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:41:58.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:41:58.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:41:58.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:497: TEST_auto_repair_bluestore_scrub: create_pool testpool 1 1 2026-02-17T05:41:58.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T05:41:59.133 INFO:tasks.workunit.client.0.trial074.stderr:pool 'testpool' created 2026-02-17T05:41:59.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:42:00.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_scrub: ceph osd pool set testpool size 2 2026-02-17T05:42:00.341 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 size to 2 2026-02-17T05:42:00.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:499: TEST_auto_repair_bluestore_scrub: wait_for_clean 2026-02-17T05:42:00.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:42:00.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:42:00.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:42:00.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:42:00.354 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:42:00.355 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:42:00.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:42:00.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:42:00.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:42:00.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:42:00.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:42:00.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:42:00.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:42:00.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:42:00.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:42:00.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:42:00.621 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:42:00.621 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:42:00.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:42:00.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:00.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:42:00.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T05:42:00.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T05:42:00.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T05:42:00.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:00.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:42:00.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T05:42:00.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T05:42:00.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T05:42:00.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:00.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:42:00.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T05:42:00.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T05:42:00.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509442' 2026-02-17T05:42:00.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:00.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T05:42:00.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:00.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:42:00.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T05:42:00.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:00.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T05:42:00.840 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:42:00.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T05:42:00.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:01.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:42:01.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:42:02.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:42:02.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:02.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:42:02.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:42:03.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:42:03.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:03.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T05:42:03.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:03.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T05:42:03.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:03.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:42:03.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T05:42:03.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:03.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T05:42:03.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T05:42:03.325 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T05:42:03.325 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:42:03.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T05:42:03.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:03.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T05:42:03.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:03.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:42:03.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T05:42:03.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:03.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T05:42:03.491 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509442 2026-02-17T05:42:03.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T05:42:03.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:42:03.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509442 -lt 64424509442 2026-02-17T05:42:03.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:42:03.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:03.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:03.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:42:03.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:03.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:03.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:03.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:03.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:03.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:03.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:03.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:42:03.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:03.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:03.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:04.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:42:04.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T05:42:04.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:42:04.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T05:42:04.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:42:04.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:42:04.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:42:04.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:04.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:04.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:04.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:04.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:04.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:04.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:04.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:42:04.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:04.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:04.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:04.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:42:04.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:42:04.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:42:04.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:42:04.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:42:04.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:42:04.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:42:04.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:42:04.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:42:04.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:42:04.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:42:04.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:42:04.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:42:04.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:42:04.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:42:04.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:04.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:04.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:04.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:04.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:04.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:04.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:05.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:42:05.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:05.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:05.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:05.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:42:05.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:42:05.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:42:05.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:42:05.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:42:05.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:42:05.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:42:05.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:42:05.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:42:05.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:42:05.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:42:05.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T05:42:05.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:42:05.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T05:42:05.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:42:05.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:05.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:05.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:05.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:05.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:05.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:05.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:06.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T05:42:06.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:06.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:06.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:06.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T05:42:06.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T05:42:06.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:42:06.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:42:06.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:42:06.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:42:06.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:42:06.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:42:06.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:42:06.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:42:06.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:42:06.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T05:42:06.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:42:06.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T05:42:07.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:42:07.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:07.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:07.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:07.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:07.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:42:07.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:07.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:07.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:502: TEST_auto_repair_bluestore_scrub: local payload=ABCDEF 2026-02-17T05:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:503: TEST_auto_repair_bluestore_scrub: echo ABCDEF 2026-02-17T05:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:504: TEST_auto_repair_bluestore_scrub: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:42:07.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:508: TEST_auto_repair_bluestore_scrub: get_not_primary testpool SOMETHING 2026-02-17T05:42:07.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:42:07.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:42:07.566 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:42:07.566 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:42:07.566 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:42:07.567 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:42:07.567 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:42:07.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:42:07.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:42:07.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:42:07.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:508: TEST_auto_repair_bluestore_scrub: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:42:07.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:42:07.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:42:07.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:42:07.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:42:07.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:42:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T05:42:09.827 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T05:42:10.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:42:10.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:42:10.367 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:10.368 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:10.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:10.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:10.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:42:10.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:42:10.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:42:10.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:42:10.372 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:42:10.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:42:10.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:42:10.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:42:10.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:42:10.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:42:10.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:42:10.391 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:10.383+0000 7f666ccd18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:10.393 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:10.383+0000 7f666ccd18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:10.394 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:10.387+0000 7f666ccd18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:10.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:42:10.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:42:10.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:42:10.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:42:10.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:42:10.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:10.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:42:10.521 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:42:10.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:10.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:10.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:11.623 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:11.615+0000 7f666ccd18c0 -1 Falling back to public interface 2026-02-17T05:42:11.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:11.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:11.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:42:11.672 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:42:11.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:11.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:11.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:12.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:12.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:12.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:42:12.826 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:42:12.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:12.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:12.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:13.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:13.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:13.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:42:13.978 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:42:13.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:13.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:14.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:14.617 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:14.607+0000 7f666ccd18c0 -1 osd.0 21 log_to_monitors true 2026-02-17T05:42:15.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:15.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:15.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:42:15.132 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:42:15.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:15.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:15.284 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 25 up_thru 0 down_at 22 last_clean_interval [5,21) [v2:127.0.0.1:6802/2982194378,v1:127.0.0.1:6803/2982194378] [v2:127.0.0.1:6804/2982194378,v1:127.0.0.1:6805/2982194378] exists,up a1a4aa42-3afc-41e8-a4f0-e334872d22c3 2026-02-17T05:42:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:42:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:42:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:42:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:42:15.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:42:15.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:42:15.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:42:15.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:42:15.285 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:42:15.285 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:42:15.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:42:15.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:42:15.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:42:15.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:42:15.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:42:15.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:42:15.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:42:15.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:42:15.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:42:15.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:42:15.547 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:42:15.547 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:42:15.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:42:15.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:15.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:42:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182402 2026-02-17T05:42:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182402 2026-02-17T05:42:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402' 2026-02-17T05:42:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:15.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:42:15.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672968 2026-02-17T05:42:15.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672968 2026-02-17T05:42:15.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402 1-42949672968' 2026-02-17T05:42:15.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:15.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:42:15.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509446 2026-02-17T05:42:15.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509446 2026-02-17T05:42:15.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402 1-42949672968 2-64424509446' 2026-02-17T05:42:15.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:15.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182402 2026-02-17T05:42:15.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:15.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:42:15.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182402 2026-02-17T05:42:15.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:15.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182402 2026-02-17T05:42:15.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182402' 2026-02-17T05:42:15.767 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 107374182402 2026-02-17T05:42:15.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:15.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 107374182402 2026-02-17T05:42:15.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:42:16.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:42:16.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:17.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182402 -lt 107374182402 2026-02-17T05:42:17.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:17.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672968 2026-02-17T05:42:17.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:17.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:42:17.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672968 2026-02-17T05:42:17.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:17.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672968 2026-02-17T05:42:17.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-17T05:42:17.081 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672968 2026-02-17T05:42:17.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:42:17.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-17T05:42:17.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:17.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509446 2026-02-17T05:42:17.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:17.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:42:17.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509446 2026-02-17T05:42:17.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509446 2026-02-17T05:42:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-17T05:42:17.243 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509446 2026-02-17T05:42:17.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:42:17.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-17T05:42:17.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:42:17.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:17.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:17.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:42:17.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:17.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:17.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:17.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:17.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:17.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:17.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:17.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:42:17.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:17.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:17.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:17.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:42:17.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:42:17.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:42:17.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:510: TEST_auto_repair_bluestore_scrub: get_pg testpool SOMETHING 2026-02-17T05:42:17.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T05:42:17.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:42:17.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-17T05:42:17.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:42:18.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:510: TEST_auto_repair_bluestore_scrub: local pgid=1.0 2026-02-17T05:42:18.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:511: TEST_auto_repair_bluestore_scrub: get_primary testpool SOMETHING 2026-02-17T05:42:18.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:42:18.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:42:18.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:42:18.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:42:18.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:511: TEST_auto_repair_bluestore_scrub: local primary=1 2026-02-17T05:42:18.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:512: TEST_auto_repair_bluestore_scrub: get_last_scrub_stamp 1.0 2026-02-17T05:42:18.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:42:18.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:42:18.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:42:18.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:42:18.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:512: TEST_auto_repair_bluestore_scrub: local last_scrub_stamp=2026-02-17T05:41:59.127808+0000 2026-02-17T05:42:18.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:513: TEST_auto_repair_bluestore_scrub: ceph tell 1.0 schedule-scrub 2026-02-17T05:42:18.425 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T05:42:18.425 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T05:42:18.425 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T05:42:18.425 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T05:40:38.420252+0000" 2026-02-17T05:42:18.425 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T05:42:18.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:516: TEST_auto_repair_bluestore_scrub: wait_for_scrub 1.0 2026-02-17T05:41:59.127808+0000 2026-02-17T05:42:18.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T05:42:18.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:41:59.127808+0000 2026-02-17T05:42:18.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:42:18.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:42:18.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:42:18.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:42:18.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:42:18.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:42:18.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:42:18.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:42:18.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:41:59.127808+0000 '>' 2026-02-17T05:41:59.127808+0000 2026-02-17T05:42:18.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:42:19.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:42:19.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:42:19.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:42:19.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:42:19.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:42:19.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:42:19.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:42:19.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:41:59.127808+0000 '>' 2026-02-17T05:41:59.127808+0000 2026-02-17T05:42:19.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:42:20.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:42:20.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:42:20.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T05:42:20.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:42:20.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:42:20.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:42:20.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:42:20.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:42:19.664856+0000 '>' 2026-02-17T05:41:59.127808+0000 2026-02-17T05:42:20.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:42:20.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:517: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T05:42:21.028 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:42:21.028 INFO:tasks.workunit.client.0.trial074.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-17T05:42:19.667046+0000 21'1 26:59 [1,0] 1 [1,0] 1 21'1 2026-02-17T05:42:19.664856+0000 21'1 2026-02-17T05:42:19.664856+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:42:19.664856+0000 1 0 2026-02-17T05:42:21.028 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:42:21.029 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:42:21.029 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:42:21.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:519: TEST_auto_repair_bluestore_scrub: sleep 2 2026-02-17T05:42:23.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:520: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T05:42:23.179 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:42:23.179 INFO:tasks.workunit.client.0.trial074.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-17T05:42:19.667046+0000 21'1 26:59 [1,0] 1 [1,0] 1 21'1 2026-02-17T05:42:19.664856+0000 21'1 2026-02-17T05:42:19.664856+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:42:19.664856+0000 1 0 2026-02-17T05:42:23.179 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:42:23.179 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:42:23.180 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:42:23.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:521: TEST_auto_repair_bluestore_scrub: sleep 2 2026-02-17T05:42:25.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:522: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T05:42:25.331 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:42:25.331 INFO:tasks.workunit.client.0.trial074.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-17T05:42:19.667046+0000 21'1 26:59 [1,0] 1 [1,0] 1 21'1 2026-02-17T05:42:19.664856+0000 21'1 2026-02-17T05:42:19.664856+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:42:19.664856+0000 1 0 2026-02-17T05:42:25.331 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:42:25.331 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:42:25.332 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:42:25.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:523: TEST_auto_repair_bluestore_scrub: sleep 5 2026-02-17T05:42:30.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:524: TEST_auto_repair_bluestore_scrub: wait_for_clean 2026-02-17T05:42:30.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:42:30.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:42:30.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:42:30.341 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:42:30.342 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:42:30.342 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:42:30.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:42:30.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:42:30.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:42:30.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:42:30.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:42:30.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:42:30.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:42:30.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:42:30.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:42:30.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:42:30.601 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:42:30.601 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:42:30.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:42:30.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:30.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:42:30.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182406 2026-02-17T05:42:30.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182406 2026-02-17T05:42:30.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182406' 2026-02-17T05:42:30.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:30.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:42:30.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672972 2026-02-17T05:42:30.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672972 2026-02-17T05:42:30.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182406 1-42949672972' 2026-02-17T05:42:30.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:30.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:42:30.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509450 2026-02-17T05:42:30.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509450 2026-02-17T05:42:30.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182406 1-42949672972 2-64424509450' 2026-02-17T05:42:30.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:30.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182406 2026-02-17T05:42:30.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:30.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:42:30.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182406 2026-02-17T05:42:30.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:30.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182406 2026-02-17T05:42:30.820 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 107374182406 2026-02-17T05:42:30.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182406' 2026-02-17T05:42:30.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:30.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182405 -lt 107374182406 2026-02-17T05:42:30.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:42:31.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:42:31.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:32.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182405 -lt 107374182406 2026-02-17T05:42:32.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:42:33.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:42:33.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:33.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-17T05:42:33.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:33.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672972 2026-02-17T05:42:33.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:33.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:42:33.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672972 2026-02-17T05:42:33.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:33.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672972 2026-02-17T05:42:33.275 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672972 2026-02-17T05:42:33.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672972' 2026-02-17T05:42:33.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:42:33.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672972 -lt 42949672972 2026-02-17T05:42:33.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:33.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509450 2026-02-17T05:42:33.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:33.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:42:33.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509450 2026-02-17T05:42:33.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:33.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509450 2026-02-17T05:42:33.435 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509450 2026-02-17T05:42:33.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2026-02-17T05:42:33.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:42:33.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509450 -lt 64424509450 2026-02-17T05:42:33.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:42:33.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:33.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:33.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:42:33.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:33.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:33.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:33.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:33.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:33.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:33.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:33.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:42:33.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:33.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:33.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:34.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:42:34.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:42:34.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:42:34.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:525: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T05:42:34.233 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:42:34.233 INFO:tasks.workunit.client.0.trial074.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-17T05:42:19.667046+0000 21'1 26:59 [1,0] 1 [1,0] 1 21'1 2026-02-17T05:42:19.664856+0000 21'1 2026-02-17T05:42:19.664856+0000 0 0 periodic scrub scheduled @ 2026-02-18T05:42:19.664856+0000 1 0 2026-02-17T05:42:34.233 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:42:34.233 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:42:34.233 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:42:34.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:528: TEST_auto_repair_bluestore_scrub: get_not_primary testpool SOMETHING 2026-02-17T05:42:34.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:42:34.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:42:34.242 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:42:34.242 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:42:34.242 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:42:34.243 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:42:34.243 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:42:34.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:42:34.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:42:34.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:42:34.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:528: TEST_auto_repair_bluestore_scrub: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:42:34.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:42:34.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:42:34.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:42:34.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:42:34.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:42:34.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:42:34.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:42:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T05:42:36.173 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:42:36.173 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:42:36.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:42:36.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:42:36.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:42:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:36.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:42:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T05:42:36.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:42:36.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:42:36.715 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:42:36.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T05:42:36.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:42:36.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:42:36.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:42:36.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:42:36.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:42:36.730 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:36.722+0000 7f4e8d1ce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:36.732 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:36.722+0000 7f4e8d1ce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:36.733 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:36.722+0000 7f4e8d1ce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:36.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:37.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:38.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:38.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:38.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:42:38.016 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:42:38.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:38.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:38.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:38.212 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:38.202+0000 7f4e8d1ce8c0 -1 Falling back to public interface 2026-02-17T05:42:39.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:39.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:39.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:42:39.168 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:42:39.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:39.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:39.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:40.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:40.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:40.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:42:40.316 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:42:40.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:40.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:40.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:41.192 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:41.182+0000 7f4e8d1ce8c0 -1 osd.0 26 log_to_monitors true 2026-02-17T05:42:41.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:41.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:41.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:42:41.465 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:42:41.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:41.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:41.613 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 30 up_thru 0 down_at 27 last_clean_interval [25,26) [v2:127.0.0.1:6802/72538228,v1:127.0.0.1:6803/72538228] [v2:127.0.0.1:6804/72538228,v1:127.0.0.1:6805/72538228] exists,up a1a4aa42-3afc-41e8-a4f0-e334872d22c3 2026-02-17T05:42:41.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:42:41.614 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:42:41.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:42:41.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:42:41.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:42:41.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:42:41.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:42:41.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:42:41.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:42:41.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:42:41.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:42:41.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:42:41.876 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:42:41.876 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:42:41.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:42:41.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:41.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:42:41.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T05:42:41.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T05:42:41.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882' 2026-02-17T05:42:41.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:41.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:42:42.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672975 2026-02-17T05:42:42.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672975 2026-02-17T05:42:42.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-42949672975' 2026-02-17T05:42:42.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:42:42.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:42:42.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509453 2026-02-17T05:42:42.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509453 2026-02-17T05:42:42.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-42949672975 2-64424509453' 2026-02-17T05:42:42.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:42.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018882 2026-02-17T05:42:42.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:42.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:42:42.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018882 2026-02-17T05:42:42.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:42.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T05:42:42.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018882' 2026-02-17T05:42:42.095 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 128849018882 2026-02-17T05:42:42.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:42.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182406 -lt 128849018882 2026-02-17T05:42:42.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:42:43.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:42:43.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:42:43.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-17T05:42:43.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:43.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672975 2026-02-17T05:42:43.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:43.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:42:43.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672975 2026-02-17T05:42:43.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:43.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672975 2026-02-17T05:42:43.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2026-02-17T05:42:43.403 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672975 2026-02-17T05:42:43.404 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:42:43.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672975 -lt 42949672975 2026-02-17T05:42:43.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:42:43.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509453 2026-02-17T05:42:43.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:42:43.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:42:43.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509453 2026-02-17T05:42:43.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:42:43.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509453 2026-02-17T05:42:43.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509453' 2026-02-17T05:42:43.558 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509453 2026-02-17T05:42:43.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:42:43.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509453 -lt 64424509453 2026-02-17T05:42:43.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:42:43.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:43.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:43.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:42:43.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:42:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:42:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:42:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:42:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:42:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:42:43.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:42:44.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:42:44.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:42:44.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:42:44.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:42:44.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:42:44.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:42:44.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:42:44.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:529: TEST_auto_repair_bluestore_scrub: rados --pool testpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:42:44.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:530: TEST_auto_repair_bluestore_scrub: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:42:44.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:531: TEST_auto_repair_bluestore_scrub: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:18.670+0000 7f22364fe640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing [ 1.0: DEEP_SCRUB_ON_ERROR ] planned DEEP_SCRUB_ON_ERROR] scrubber: scrub_finish before flags: DEEP_SCRUB_ON_ERROR. repair state: no-repair. deep_scrub_on_error: 1 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:18.670+0000 7f22364fe640 15 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing [ 1.0: DEEP_SCRUB_ON_ERROR ] ] scrubber: scrub_finish Try to auto repair after scrub errors 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:18.670+0000 7f22364fe640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing [ 1.0: ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:19.658+0000 7f22364fe640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:19.658+0000 7f22364fe640 10 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:19.658+0000 7f22364fe640 15 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:19.658+0000 7f22364fe640 20 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-17T05:42:44.251 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:42:19.658+0000 7f22364fe640 19 osd.1 pg_epoch: 26 pg[1.0( v 21'1 (0'0,21'1] local-lis/les=25/26 n=1 ec=17/17 lis/c=25/25 les/c/f=26/26/0 sis=25) [1,0] r=0 lpr=25 crt=21'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:42:44.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:534: TEST_auto_repair_bluestore_scrub: ceph pg 1.0 query 2026-02-17T05:42:44.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:534: TEST_auto_repair_bluestore_scrub: jq .info.stats.stat_sum.num_objects_repaired 2026-02-17T05:42:44.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:534: TEST_auto_repair_bluestore_scrub: COUNT=1 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:535: TEST_auto_repair_bluestore_scrub: test 1 = 1 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:42:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:42:44.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:42:44.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:42:44.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:42:44.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:42:44.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:42:44.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:42:44.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:42:44.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:42:44.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:42:44.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:42:44.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:42:44.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:42:44.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:42:44.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:42:44.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:42:44.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:42:44.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:42:44.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:42:44.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:42:44.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:42:44.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:42:44.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:42:44.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:42:44.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:42:44.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:42:44.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:42:44.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:42:44.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:42:44.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:42:44.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:42:44.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:42:44.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:42:44.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:42:44.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:42:44.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:42:44.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:42:44.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:42:44.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:42:44.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:42:44.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:42:44.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:42:44.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:42:44.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:42:44.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:42:44.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_tag td/osd-scrub-repair 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:383: TEST_auto_repair_bluestore_tag: local dir=td/osd-scrub-repair 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:384: TEST_auto_repair_bluestore_tag: local poolname=testpool 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:387: TEST_auto_repair_bluestore_tag: run_mon td/osd-scrub-repair a 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:42:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:42:44.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:42:44.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:42:44.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:42:44.497 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:42:44.497 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.497 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:44.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:42:44.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:42:44.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:42:44.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:42:44.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:42:44.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:42:44.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:42:44.527 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:42:44.527 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:42:44.527 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:42:44.527 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:42:44.527 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.527 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.527 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:42:44.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:42:44.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:42:44.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:42:44.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.583 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.584 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:42:44.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:42:44.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:42:44.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:388: TEST_auto_repair_bluestore_tag: run_mgr td/osd-scrub-repair x 2026-02-17T05:42:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:42:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:42:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:42:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:42:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:42:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:42:44.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:42:44.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:42:44.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:42:44.734 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:42:44.734 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.734 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:44.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:42:44.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:42:44.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:395: TEST_auto_repair_bluestore_tag: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T05:42:44.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: seq 0 2 2026-02-17T05:42:44.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-17T05:42:44.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:397: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:42:44.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:42:44.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:42:44.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:42:44.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:42:44.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:42:44.760 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:42:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:42:44.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:42:44.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:42:44.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T05:42:44.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:42:44.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:42:44.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a8641ef1-a75d-413d-b693-c138b73925cc 2026-02-17T05:42:44.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a8641ef1-a75d-413d-b693-c138b73925cc' 2026-02-17T05:42:44.763 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 a8641ef1-a75d-413d-b693-c138b73925cc 2026-02-17T05:42:44.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:42:44.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDU/5NpyYKbLRAAvCnYi/6hE5JosFhQxX5AUw== 2026-02-17T05:42:44.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDU/5NpyYKbLRAAvCnYi/6hE5JosFhQxX5AUw=="}' 2026-02-17T05:42:44.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a8641ef1-a75d-413d-b693-c138b73925cc -i td/osd-scrub-repair/0/new.json 2026-02-17T05:42:44.862 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:42:44.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:42:44.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQDU/5NpyYKbLRAAvCnYi/6hE5JosFhQxX5AUw== --osd-uuid a8641ef1-a75d-413d-b693-c138b73925cc 2026-02-17T05:42:44.888 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:44.878+0000 7f3481e9f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:44.889 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:44.878+0000 7f3481e9f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:44.890 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:44.878+0000 7f3481e9f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:44.890 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:44.882+0000 7f3481e9f8c0 -1 bdev(0x55ef03a53c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:42:44.890 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:44.882+0000 7f3481e9f8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:42:47.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:42:47.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:42:47.505 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:42:47.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:42:47.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:42:47.697 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:42:47.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:42:47.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:42:47.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:42:47.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:42:47.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:42:47.724 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:47.714+0000 7f4e8706f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:47.725 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:47.714+0000 7f4e8706f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:47.727 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:47.718+0000 7f4e8706f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:47.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:47.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:47.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:48.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:48.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:48.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:42:49.000 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:42:49.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:49.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:49.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:49.208 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:49.197+0000 7f4e8706f8c0 -1 Falling back to public interface 2026-02-17T05:42:50.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:50.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:50.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:42:50.154 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:42:50.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:50.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:50.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:51.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:42:51.307 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:42:51.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:51.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:51.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:52.210 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:52.197+0000 7f4e8706f8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:42:52.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:52.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:52.457 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:42:52.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:42:52.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:52.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:42:52.604 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2561321023,v1:127.0.0.1:6803/2561321023] [v2:127.0.0.1:6804/2561321023,v1:127.0.0.1:6805/2561321023] exists,up a8641ef1-a75d-413d-b693-c138b73925cc 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:397: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:42:52.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:42:52.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:42:52.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:42:52.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=fa6a2c08-51b3-4f70-9d74-8a75ac2a086d 2026-02-17T05:42:52.608 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 fa6a2c08-51b3-4f70-9d74-8a75ac2a086d 2026-02-17T05:42:52.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 fa6a2c08-51b3-4f70-9d74-8a75ac2a086d' 2026-02-17T05:42:52.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:42:52.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDc/5NpVpKfJBAA68W7YPLq8M0DN5S4TKDnfQ== 2026-02-17T05:42:52.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDc/5NpVpKfJBAA68W7YPLq8M0DN5S4TKDnfQ=="}' 2026-02-17T05:42:52.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new fa6a2c08-51b3-4f70-9d74-8a75ac2a086d -i td/osd-scrub-repair/1/new.json 2026-02-17T05:42:52.773 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:42:52.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:42:52.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQDc/5NpVpKfJBAA68W7YPLq8M0DN5S4TKDnfQ== --osd-uuid fa6a2c08-51b3-4f70-9d74-8a75ac2a086d 2026-02-17T05:42:52.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:52.797+0000 7fdf574378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:52.810 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:52.797+0000 7fdf574378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:52.811 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:52.801+0000 7fdf574378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:52.811 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:52.801+0000 7fdf574378c0 -1 bdev(0x55bc05003c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:42:52.811 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:52.801+0000 7fdf574378c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:42:55.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:42:55.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:42:55.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:42:55.102 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:42:55.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:42:55.292 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:42:55.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:42:55.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:42:55.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:42:55.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:42:55.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:42:55.316 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:55.305+0000 7fb94a3df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:55.318 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:55.305+0000 7fb94a3df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:55.319 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:55.309+0000 7fb94a3df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:42:55.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:42:55.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:42:55.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:42:55.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:42:55.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:42:55.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:55.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:42:55.445 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:42:55.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:55.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:42:55.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:56.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:56.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:56.596 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:42:56.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:42:56.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:56.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:42:56.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:56.804 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:56.793+0000 7fb94a3df8c0 -1 Falling back to public interface 2026-02-17T05:42:57.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:57.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:57.747 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:42:57.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:42:57.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:57.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:42:57.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:58.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:42:58.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:42:58.897 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:42:58.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:42:58.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:42:58.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:42:59.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:42:59.795 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:42:59.785+0000 7fb94a3df8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:43:00.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:00.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:00.053 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:43:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:43:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:43:00.203 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1838648427,v1:127.0.0.1:6811/1838648427] [v2:127.0.0.1:6812/1838648427,v1:127.0.0.1:6813/1838648427] exists,up fa6a2c08-51b3-4f70-9d74-8a75ac2a086d 2026-02-17T05:43:00.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:43:00.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:43:00.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:43:00.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-17T05:43:00.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:397: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:00.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:00.204 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T05:43:00.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:43:00.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:43:00.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a31a5dab-9e27-4c8c-97a5-ad0b91bd56bb 2026-02-17T05:43:00.208 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 a31a5dab-9e27-4c8c-97a5-ad0b91bd56bb 2026-02-17T05:43:00.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a31a5dab-9e27-4c8c-97a5-ad0b91bd56bb' 2026-02-17T05:43:00.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:43:00.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDk/5NpkGi5DBAAcuAu41CKBuAXgminVrGtSA== 2026-02-17T05:43:00.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDk/5NpkGi5DBAAcuAu41CKBuAXgminVrGtSA=="}' 2026-02-17T05:43:00.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a31a5dab-9e27-4c8c-97a5-ad0b91bd56bb -i td/osd-scrub-repair/2/new.json 2026-02-17T05:43:00.366 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:43:00.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:43:00.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQDk/5NpkGi5DBAAcuAu41CKBuAXgminVrGtSA== --osd-uuid a31a5dab-9e27-4c8c-97a5-ad0b91bd56bb 2026-02-17T05:43:00.392 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:00.381+0000 7f42cdfb58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:00.394 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:00.381+0000 7f42cdfb58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:00.395 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:00.385+0000 7f42cdfb58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:00.396 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:00.385+0000 7f42cdfb58c0 -1 bdev(0x55ff5420bc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:43:00.396 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:00.385+0000 7f42cdfb58c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:43:02.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:43:02.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:43:02.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:43:02.719 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:43:02.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:43:02.899 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:43:02.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:43:02.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:02.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:43:02.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:43:02.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:43:02.924 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:02.913+0000 7f7b70a6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:02.926 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:02.913+0000 7f7b70a6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:02.927 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:02.917+0000 7f7b70a6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:03.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:43:03.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:43:03.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:43:03.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:43:03.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:43:03.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:03.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:43:03.054 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:43:03.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:03.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:43:03.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:04.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:04.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:04.203 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:43:04.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:43:04.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:04.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:43:04.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:04.428 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:04.417+0000 7f7b70a6c8c0 -1 Falling back to public interface 2026-02-17T05:43:05.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:05.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:05.351 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:43:05.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:43:05.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:05.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:43:05.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:06.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:06.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:06.499 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:43:06.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:43:06.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:06.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:43:06.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:07.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:07.401+0000 7f7b70a6c8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:43:07.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:07.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:07.648 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:43:07.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:43:07.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:07.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:43:07.795 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2661758606,v1:127.0.0.1:6819/2661758606] [v2:127.0.0.1:6820/2661758606,v1:127.0.0.1:6821/2661758606] exists,up a31a5dab-9e27-4c8c-97a5-ad0b91bd56bb 2026-02-17T05:43:07.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:43:07.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:43:07.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:43:07.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:400: TEST_auto_repair_bluestore_tag: create_pool testpool 1 1 2026-02-17T05:43:07.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T05:43:07.974 INFO:tasks.workunit.client.0.trial074.stderr:pool 'testpool' created 2026-02-17T05:43:07.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:43:08.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:401: TEST_auto_repair_bluestore_tag: ceph osd pool set testpool size 2 2026-02-17T05:43:09.200 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 size to 2 2026-02-17T05:43:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:402: TEST_auto_repair_bluestore_tag: wait_for_clean 2026-02-17T05:43:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:43:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:43:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:43:09.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:43:09.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:43:09.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:43:09.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:43:09.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:43:09.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:43:09.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:43:09.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:43:09.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:43:09.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:43:09.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:43:09.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:43:09.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:43:09.477 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:09.477 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:43:09.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:43:09.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:09.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:43:09.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T05:43:09.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T05:43:09.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T05:43:09.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:09.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:43:09.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705667 2026-02-17T05:43:09.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705667 2026-02-17T05:43:09.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705667' 2026-02-17T05:43:09.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:09.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:43:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T05:43:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T05:43:09.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705667 2-60129542146' 2026-02-17T05:43:09.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:09.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T05:43:09.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:09.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:43:09.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T05:43:09.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:09.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T05:43:09.696 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:43:09.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T05:43:09.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:09.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:43:09.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:10.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:43:10.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:11.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:43:11.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:12.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:43:12.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:12.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-17T05:43:12.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:12.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705667 2026-02-17T05:43:12.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:12.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:43:12.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705667 2026-02-17T05:43:12.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:12.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705667 2026-02-17T05:43:12.168 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705667 2026-02-17T05:43:12.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-17T05:43:12.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:43:12.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-17T05:43:12.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:12.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T05:43:12.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:12.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:43:12.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T05:43:12.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:12.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T05:43:12.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T05:43:12.322 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542146 2026-02-17T05:43:12.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:43:12.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T05:43:12.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:43:12.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:12.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:12.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:43:12.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:43:12.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:43:12.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:43:12.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:43:12.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:43:12.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:43:12.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:43:12.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:43:12.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:43:12.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:12.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:12.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:43:12.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:43:12.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:43:12.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:405: TEST_auto_repair_bluestore_tag: local payload=ABCDEF 2026-02-17T05:43:12.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:406: TEST_auto_repair_bluestore_tag: echo ABCDEF 2026-02-17T05:43:12.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:407: TEST_auto_repair_bluestore_tag: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:43:13.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:411: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-17T05:43:13.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:43:13.012 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:43:13.012 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:43:13.012 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:43:13.012 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:43:13.012 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:13.013 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:43:13.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:43:13.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:13.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:411: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:43:13.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:43:13.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:43:14.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:43:14.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:43:14.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:43:14.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:43:14.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:43:14.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:43:14.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:43:14.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T05:43:15.269 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:43:15.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:15.827 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T05:43:15.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:43:15.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:43:15.845 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:15.833+0000 7f36838198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:15.846 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:15.833+0000 7f36838198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:15.847 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:15.837+0000 7f36838198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:15.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:43:15.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:43:15.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:43:15.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:43:15.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:43:15.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:15.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:43:15.980 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:43:15.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:15.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:16.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:17.057 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:17.045+0000 7f36838198c0 -1 Falling back to public interface 2026-02-17T05:43:17.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:17.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:17.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:43:17.135 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:17.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:17.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:17.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:18.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:18.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:18.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:43:18.287 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:43:18.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:18.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:18.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:19.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:19.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:19.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:43:19.439 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:43:19.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:19.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:19.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:20.020 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:20.009+0000 7f36838198c0 -1 osd.0 19 log_to_monitors true 2026-02-17T05:43:20.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:20.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:20.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:43:20.590 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:43:20.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:20.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:20.743 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/2629222665,v1:127.0.0.1:6803/2629222665] [v2:127.0.0.1:6804/2629222665,v1:127.0.0.1:6805/2629222665] exists,up a8641ef1-a75d-413d-b693-c138b73925cc 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:43:20.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:43:20.745 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:43:20.745 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:43:20.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:43:20.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:43:20.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:43:20.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:43:20.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:43:20.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:43:20.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:43:20.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:43:20.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:43:21.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:43:21.003 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:21.003 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:43:21.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:43:21.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:21.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:43:21.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T05:43:21.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T05:43:21.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T05:43:21.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:21.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:43:21.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-17T05:43:21.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-17T05:43:21.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705671' 2026-02-17T05:43:21.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:21.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:43:21.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542149 2026-02-17T05:43:21.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542149 2026-02-17T05:43:21.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705671 2-60129542149' 2026-02-17T05:43:21.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:21.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T05:43:21.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:21.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:43:21.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T05:43:21.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:21.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T05:43:21.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T05:43:21.222 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 98784247810 2026-02-17T05:43:21.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:21.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 98784247810 2026-02-17T05:43:21.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:22.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:43:22.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:22.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 98784247810 2026-02-17T05:43:22.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:23.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:43:23.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:23.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T05:43:23.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:23.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-17T05:43:23.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:23.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:43:23.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-17T05:43:23.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:23.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-17T05:43:23.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T05:43:23.691 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705671 2026-02-17T05:43:23.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:43:23.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T05:43:23.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:23.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542149 2026-02-17T05:43:23.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:23.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:43:23.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542149 2026-02-17T05:43:23.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:23.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542149 2026-02-17T05:43:23.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-17T05:43:23.858 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542149 2026-02-17T05:43:23.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:43:24.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542149 -lt 60129542149 2026-02-17T05:43:24.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:43:24.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:24.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:43:24.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:43:24.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:43:24.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:43:24.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:24.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:24.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:43:24.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:43:24.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:43:24.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:413: TEST_auto_repair_bluestore_tag: get_pg testpool SOMETHING 2026-02-17T05:43:24.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T05:43:24.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:43:24.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:24.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:43:24.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:413: TEST_auto_repair_bluestore_tag: local pgid=1.0 2026-02-17T05:43:24.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:414: TEST_auto_repair_bluestore_tag: get_primary testpool SOMETHING 2026-02-17T05:43:24.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:43:24.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:43:24.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:24.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:43:24.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:414: TEST_auto_repair_bluestore_tag: local primary=1 2026-02-17T05:43:24.811 INFO:tasks.workunit.client.0.trial074.stdout:Affected PG 1.0 w/ primary 1 2026-02-17T05:43:24.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:415: TEST_auto_repair_bluestore_tag: echo 'Affected PG ' 1.0 ' w/ primary ' 1 2026-02-17T05:43:24.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:416: TEST_auto_repair_bluestore_tag: get_last_scrub_stamp 1.0 2026-02-17T05:43:24.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:43:24.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:43:24.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:43:24.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:43:24.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:416: TEST_auto_repair_bluestore_tag: local last_scrub_stamp=2026-02-17T05:43:07.965625+0000 2026-02-17T05:43:24.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:417: TEST_auto_repair_bluestore_tag: initiate_and_fetch_state 1 1.0 3.0 2026-02-17T05:43:24.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:322: initiate_and_fetch_state: local the_osd=osd.1 2026-02-17T05:43:24.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:323: initiate_and_fetch_state: local pgid=1.0 2026-02-17T05:43:24.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:324: initiate_and_fetch_state: get_last_scrub_stamp 1.0 2026-02-17T05:43:24.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T05:43:24.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:43:24.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:43:24.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T05:43:25.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:324: initiate_and_fetch_state: local last_scrub=2026-02-17T05:43:07.965625+0000 2026-02-17T05:43:25.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:326: initiate_and_fetch_state: set_config osd 1 osd_scrub_sleep 3.0 2026-02-17T05:43:25.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T05:43:25.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T05:43:25.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=osd_scrub_sleep 2026-02-17T05:43:25.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=3.0 2026-02-17T05:43:25.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T05:43:25.118 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T05:43:25.118 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T05:43:25.118 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T05:43:25.118 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:43:25.119 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:25.119 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:25.119 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T05:43:25.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.1.asok config set osd_scrub_sleep 3.0 2026-02-17T05:43:25.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:327: initiate_and_fetch_state: set_config osd 1 osd_scrub_auto_repair true 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=osd_scrub_auto_repair 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T05:43:25.178 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T05:43:25.179 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T05:43:25.179 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:43:25.179 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:25.179 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:25.179 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T05:43:25.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.1.asok config set osd_scrub_auto_repair true 2026-02-17T05:43:25.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T05:43:25.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:329: initiate_and_fetch_state: flush_pg_stats 2026-02-17T05:43:25.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:43:25.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:43:25.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:43:25.391 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:25.391 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:43:25.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:43:25.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:25.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:43:25.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247812 2026-02-17T05:43:25.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247812 2026-02-17T05:43:25.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247812' 2026-02-17T05:43:25.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:25.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:43:25.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-17T05:43:25.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-17T05:43:25.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247812 1-38654705672' 2026-02-17T05:43:25.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:25.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:43:25.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542151 2026-02-17T05:43:25.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542151 2026-02-17T05:43:25.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247812 1-38654705672 2-60129542151' 2026-02-17T05:43:25.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:25.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247812 2026-02-17T05:43:25.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:25.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:43:25.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247812 2026-02-17T05:43:25.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:25.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247812 2026-02-17T05:43:25.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247812' 2026-02-17T05:43:25.608 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 98784247812 2026-02-17T05:43:25.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:25.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247812 2026-02-17T05:43:25.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:26.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:43:26.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:26.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247812 2026-02-17T05:43:26.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:27.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:43:27.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:28.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247812 -lt 98784247812 2026-02-17T05:43:28.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:28.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-17T05:43:28.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:28.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:43:28.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-17T05:43:28.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:28.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-17T05:43:28.078 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705672 2026-02-17T05:43:28.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-17T05:43:28.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:43:28.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705673 -lt 38654705672 2026-02-17T05:43:28.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:28.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542151 2026-02-17T05:43:28.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:28.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:43:28.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542151 2026-02-17T05:43:28.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:28.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542151 2026-02-17T05:43:28.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-17T05:43:28.240 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542151 2026-02-17T05:43:28.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:43:28.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542151 -lt 60129542151 2026-02-17T05:43:28.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:330: initiate_and_fetch_state: date --rfc-3339=ns 2026-02-17T05:43:28.393 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17 05:43:28.386618762+00:00 2026-02-17T05:43:28.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:333: initiate_and_fetch_state: get_asok_path osd.1 2026-02-17T05:43:28.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T05:43:28.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T05:43:28.394 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:43:28.395 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:28.395 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:28.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T05:43:28.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:333: initiate_and_fetch_state: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.1.asok schedule-deep-scrub 1.0 2026-02-17T05:43:28.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i=0 )) 2026-02-17T05:43:28.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:28.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:28.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:28.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:28.531 INFO:tasks.workunit.client.0.trial074.stdout:{"deep":true,"must":false,"stamp":"2026-02-03T05:41:48.446222+0000"}0 ) state now: "active+clean" 2026-02-17T05:43:28.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 0 ') state now: ' '"active+clean"' 2026-02-17T05:43:28.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:28.532 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:28.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:28.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 0 == 4 ']' 2026-02-17T05:43:28.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:28.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:28.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:28.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:28.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:28.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:28.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 1 ') state now: ' '"active+clean"' 2026-02-17T05:43:28.906 INFO:tasks.workunit.client.0.trial074.stdout:1 ) state now: "active+clean" 2026-02-17T05:43:28.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:28.906 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:28.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:28.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 1 == 4 ']' 2026-02-17T05:43:28.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:29.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:29.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:29.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:29.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:29.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:29.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 2 ') state now: ' '"active+clean"' 2026-02-17T05:43:29.279 INFO:tasks.workunit.client.0.trial074.stdout:2 ) state now: "active+clean" 2026-02-17T05:43:29.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:29.280 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:29.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:29.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 2 == 4 ']' 2026-02-17T05:43:29.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:29.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:29.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:29.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:29.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:29.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:29.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 3 ') state now: ' '"active+clean"' 2026-02-17T05:43:29.652 INFO:tasks.workunit.client.0.trial074.stdout:3 ) state now: "active+clean" 2026-02-17T05:43:29.652 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:29.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:29.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:29.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 3 == 4 ']' 2026-02-17T05:43:29.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:29.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:29.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:29.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:29.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:30.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:30.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 4 ') state now: ' '"active+clean"' 2026-02-17T05:43:30.026 INFO:tasks.workunit.client.0.trial074.stdout:4 ) state now: "active+clean" 2026-02-17T05:43:30.026 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:30.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:30.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:30.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 4 == 4 ']' 2026-02-17T05:43:30.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:350: initiate_and_fetch_state: echo 'loop --------> ' 4 2026-02-17T05:43:30.027 INFO:tasks.workunit.client.0.trial074.stdout:loop --------> 4 2026-02-17T05:43:30.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:30.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:30.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:30.328 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:30.328 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:30.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:30.401 INFO:tasks.workunit.client.0.trial074.stdout:5 ) state now: "active+clean" 2026-02-17T05:43:30.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 5 ') state now: ' '"active+clean"' 2026-02-17T05:43:30.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:30.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:30.403 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:30.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 5 == 4 ']' 2026-02-17T05:43:30.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:30.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:30.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:30.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:30.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:30.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:30.776 INFO:tasks.workunit.client.0.trial074.stdout:6 ) state now: "active+clean" 2026-02-17T05:43:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 6 ') state now: ' '"active+clean"' 2026-02-17T05:43:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:30.776 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:30.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:30.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 6 == 4 ']' 2026-02-17T05:43:30.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:31.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:31.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:31.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:31.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:31.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:31.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 7 ') state now: ' '"active+clean"' 2026-02-17T05:43:31.149 INFO:tasks.workunit.client.0.trial074.stdout:7 ) state now: "active+clean" 2026-02-17T05:43:31.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:31.149 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 7 == 4 ']' 2026-02-17T05:43:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:31.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:31.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:31.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:31.523 INFO:tasks.workunit.client.0.trial074.stdout:8 ) state now: "active+clean" 2026-02-17T05:43:31.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 8 ') state now: ' '"active+clean"' 2026-02-17T05:43:31.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:31.524 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:31.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:31.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 8 == 4 ']' 2026-02-17T05:43:31.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:31.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:31.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:31.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:31.898 INFO:tasks.workunit.client.0.trial074.stdout:9 ) state now: "active+clean" 2026-02-17T05:43:31.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 9 ') state now: ' '"active+clean"' 2026-02-17T05:43:31.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:31.898 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:31.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:31.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 9 == 4 ']' 2026-02-17T05:43:31.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:32.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:32.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:32.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:32.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:32.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T05:43:32.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 10 ') state now: ' '"active+clean"' 2026-02-17T05:43:32.275 INFO:tasks.workunit.client.0.trial074.stdout:10 ) state now: "active+clean" 2026-02-17T05:43:32.275 INFO:tasks.workunit.client.0.trial074.stdout:"active+clean" 2026-02-17T05:43:32.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:32.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T05:43:32.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 0 == 4 ']' 2026-02-17T05:43:32.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T05:43:32.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T05:43:32.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T05:43:32.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T05:43:32.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T05:43:32.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean+scrubbing+deep"' 2026-02-17T05:43:32.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 11 ') state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T05:43:32.650 INFO:tasks.workunit.client.0.trial074.stdout:11 ) state now: "active+clean+scrubbing+deep" 2026-02-17T05:43:32.650 INFO:tasks.workunit.client.0.trial074.stdout:found scrub 2026-02-17T05:43:32.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T05:43:32.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:343: initiate_and_fetch_state: echo 'found scrub' 2026-02-17T05:43:32.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:343: initiate_and_fetch_state: return 0 2026-02-17T05:43:32.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:418: TEST_auto_repair_bluestore_tag: r=0 2026-02-17T05:43:32.651 INFO:tasks.workunit.client.0.trial074.stdout:initiate_and_fetch_state ret: 0 2026-02-17T05:43:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:419: TEST_auto_repair_bluestore_tag: echo 'initiate_and_fetch_state ret: ' 0 2026-02-17T05:43:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:420: TEST_auto_repair_bluestore_tag: set_config osd td/osd-scrub-repair osd_scrub_sleep 0 2026-02-17T05:43:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T05:43:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=td/osd-scrub-repair 2026-02-17T05:43:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=osd_scrub_sleep 2026-02-17T05:43:32.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=0 2026-02-17T05:43:32.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.td/osd-scrub-repair 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.td/osd-scrub-repair 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.td/osd-scrub-repair ']' 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.td/osd-scrub-repair.asok 2026-02-17T05:43:32.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.td/osd-scrub-repair.asok config set osd_scrub_sleep 0 2026-02-17T05:43:32.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test == true 2026-02-17T05:43:32.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh: line 1168: test: ==: unary operator expected 2026-02-17T05:43:32.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:421: TEST_auto_repair_bluestore_tag: '[' 0 -ne 0 ']' 2026-02-17T05:43:32.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: TEST_auto_repair_bluestore_tag: wait_end_of_scrub 1 1.0 2026-02-17T05:43:32.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:360: wait_end_of_scrub: local the_osd=osd.1 2026-02-17T05:43:32.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:361: wait_end_of_scrub: local pgid=1.0 2026-02-17T05:43:32.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i=0 )) 2026-02-17T05:43:32.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T05:43:32.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T05:43:32.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T05:43:32.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T05:43:32.769 INFO:tasks.workunit.client.0.trial074.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T05:43:32.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T05:43:32.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T05:43:32.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 0 == 4 ']' 2026-02-17T05:43:32.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T05:43:33.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T05:43:33.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T05:43:33.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T05:43:33.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T05:43:33.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T05:43:33.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T05:43:33.142 INFO:tasks.workunit.client.0.trial074.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T05:43:33.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T05:43:33.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 1 == 4 ']' 2026-02-17T05:43:33.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T05:43:33.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T05:43:33.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T05:43:33.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T05:43:33.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T05:43:33.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T05:43:33.515 INFO:tasks.workunit.client.0.trial074.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T05:43:33.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T05:43:33.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T05:43:33.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 2 == 4 ']' 2026-02-17T05:43:33.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T05:43:33.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T05:43:33.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T05:43:33.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T05:43:33.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T05:43:33.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T05:43:33.894 INFO:tasks.workunit.client.0.trial074.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T05:43:33.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T05:43:33.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T05:43:33.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 3 == 4 ']' 2026-02-17T05:43:33.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T05:43:34.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T05:43:34.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T05:43:34.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T05:43:34.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T05:43:34.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T05:43:34.272 INFO:tasks.workunit.client.0.trial074.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T05:43:34.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T05:43:34.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T05:43:34.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 4 == 4 ']' 2026-02-17T05:43:34.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:368: wait_end_of_scrub: flush_pg_stats 2026-02-17T05:43:34.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:43:34.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:43:34.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:43:34.421 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:34.421 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:43:34.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:43:34.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:34.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:43:34.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247814 2026-02-17T05:43:34.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247814 2026-02-17T05:43:34.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247814' 2026-02-17T05:43:34.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:34.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:43:34.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T05:43:34.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T05:43:34.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247814 1-38654705675' 2026-02-17T05:43:34.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:34.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:43:34.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542154 2026-02-17T05:43:34.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542154 2026-02-17T05:43:34.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247814 1-38654705675 2-60129542154' 2026-02-17T05:43:34.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:34.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247814 2026-02-17T05:43:34.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:34.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:43:34.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247814 2026-02-17T05:43:34.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:34.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247814 2026-02-17T05:43:34.642 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 98784247814 2026-02-17T05:43:34.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247814' 2026-02-17T05:43:34.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:34.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247813 -lt 98784247814 2026-02-17T05:43:34.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:35.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:43:35.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:35.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247814 -lt 98784247814 2026-02-17T05:43:35.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:35.957 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T05:43:35.957 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:35.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:43:35.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T05:43:35.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:35.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T05:43:35.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T05:43:35.961 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705675 2026-02-17T05:43:35.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:43:36.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T05:43:36.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:36.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542154 2026-02-17T05:43:36.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:36.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:43:36.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542154 2026-02-17T05:43:36.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:36.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542154 2026-02-17T05:43:36.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542154' 2026-02-17T05:43:36.112 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542154 2026-02-17T05:43:36.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:43:36.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542154 -lt 60129542154 2026-02-17T05:43:36.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T05:43:36.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T05:43:36.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T05:43:36.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T05:43:36.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T05:43:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean"' 2026-02-17T05:43:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean"' 2026-02-17T05:43:36.639 INFO:tasks.workunit.client.0.trial074.stdout:wait-scrub-end state now: "active+clean" 2026-02-17T05:43:36.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean" =~ (.*scrubbing.*) ]] 2026-02-17T05:43:36.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: break 2026-02-17T05:43:36.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:373: wait_end_of_scrub: [[ "active+clean" =~ (.*scrubbing.*) ]] 2026-02-17T05:43:36.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:378: wait_end_of_scrub: return 0 2026-02-17T05:43:36.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: TEST_auto_repair_bluestore_tag: ceph pg dump pgs 2026-02-17T05:43:36.778 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T05:43:36.778 INFO:tasks.workunit.client.0.trial074.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean+scrubbing+deep 2026-02-17T05:43:32.627601+0000 19'1 25:46 [1,0] 1 [1,0] 1 0'0 2026-02-03T05:41:48.446222+0000 0'0 2026-02-03T05:41:48.446222+0000 0 0 deep scrubbing for 2s 0 0 2026-02-17T05:43:36.778 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T05:43:36.778 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T05:43:36.779 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T05:43:36.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-17T05:43:36.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:43:36.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:43:36.788 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:43:36.788 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:43:36.788 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:43:36.788 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:36.789 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:43:36.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:43:36.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:36.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:43:37.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:43:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:43:37.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:43:37.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:43:37.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:43:37.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:43:37.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:43:38.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:43:38.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T05:43:38.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:43:38.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:43:38.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:43:38.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:43:38.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:43:38.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T05:43:38.710 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:43:38.711 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:43:39.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:39.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:43:39.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:43:39.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:43:39.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:43:39.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:43:39.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:43:39.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:43:39.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:39.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:39.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:39.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:43:39.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:43:39.251 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:43:39.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:39.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:43:39.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:43:39.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:43:39.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:43:39.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:43:39.267 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:39.256+0000 7f92fbee18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:39.269 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:39.260+0000 7f92fbee18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:39.270 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:39.260+0000 7f92fbee18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:39.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:43:39.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:43:39.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:43:39.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:43:39.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:43:39.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:39.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:43:39.399 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:43:39.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:39.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:39.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:40.497 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:40.488+0000 7f92fbee18c0 -1 Falling back to public interface 2026-02-17T05:43:40.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:40.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:40.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:43:40.551 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:40.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:40.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:40.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:41.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:41.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:41.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:43:41.704 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:43:41.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:41.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:41.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:42.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:42.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:42.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:43:42.854 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:43:42.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:42.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:43.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:43.495 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:43.484+0000 7f92fbee18c0 -1 osd.0 25 log_to_monitors true 2026-02-17T05:43:44.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:44.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:44.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:43:44.007 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:43:44.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:44.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:44.159 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 29 up_thru 0 down_at 26 last_clean_interval [23,25) [v2:127.0.0.1:6802/3060776911,v1:127.0.0.1:6803/3060776911] [v2:127.0.0.1:6804/3060776911,v1:127.0.0.1:6805/3060776911] exists,up a8641ef1-a75d-413d-b693-c138b73925cc 2026-02-17T05:43:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:43:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:43:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:43:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:43:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:43:44.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:43:44.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:43:44.160 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:43:44.161 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:43:44.161 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:43:44.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:43:44.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:43:44.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:43:44.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:43:44.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:43:44.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:43:44.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:43:44.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:43:44.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:43:44.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:43:44.420 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:44.421 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:43:44.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:43:44.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:44.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:43:44.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T05:43:44.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T05:43:44.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586' 2026-02-17T05:43:44.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:44.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:43:44.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705678 2026-02-17T05:43:44.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705678 2026-02-17T05:43:44.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705678' 2026-02-17T05:43:44.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:44.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:43:44.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542157 2026-02-17T05:43:44.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542157 2026-02-17T05:43:44.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705678 2-60129542157' 2026-02-17T05:43:44.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:44.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051586 2026-02-17T05:43:44.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:44.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:43:44.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051586 2026-02-17T05:43:44.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T05:43:44.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-17T05:43:44.640 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 124554051586 2026-02-17T05:43:44.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:44.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247815 -lt 124554051586 2026-02-17T05:43:44.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:45.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:43:45.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:45.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T05:43:45.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:45.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705678 2026-02-17T05:43:45.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:45.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:43:45.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705678 2026-02-17T05:43:45.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:45.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705678 2026-02-17T05:43:45.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705678' 2026-02-17T05:43:45.951 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705678 2026-02-17T05:43:45.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:43:46.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705678 -lt 38654705678 2026-02-17T05:43:46.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:46.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542157 2026-02-17T05:43:46.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:46.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:43:46.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542157 2026-02-17T05:43:46.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:46.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542157 2026-02-17T05:43:46.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-17T05:43:46.116 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542157 2026-02-17T05:43:46.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:43:46.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-17T05:43:46.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:43:46.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:46.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:43:46.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:43:46.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:43:46.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:43:46.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:46.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:46.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:43:46.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:43:46.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:43:46.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:431: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-17T05:43:46.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T05:43:46.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:43:46.782 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T05:43:46.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T05:43:46.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:43:46.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:46.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:43:46.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:43:46.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T05:43:46.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:431: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:43:47.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:43:47.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:43:48.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:43:48.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:43:48.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:43:48.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:43:48.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:43:48.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:43:48.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:43:48.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T05:43:49.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:49.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:43:49.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:43:49.250 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:43:49.250 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:49.250 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:49.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T05:43:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:43:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:43:49.252 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:43:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T05:43:49.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:43:49.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:43:49.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:43:49.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:43:49.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:43:49.276 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:49.268+0000 7f6b2e5188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:49.277 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:49.268+0000 7f6b2e5188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:49.279 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:49.268+0000 7f6b2e5188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:49.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:43:49.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:49.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:50.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:50.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:50.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:43:50.561 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:50.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:50.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:50.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:51.046 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:51.036+0000 7f6b2e5188c0 -1 Falling back to public interface 2026-02-17T05:43:51.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:51.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:51.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:43:51.711 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:43:51.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:51.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:51.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:52.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:52.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:52.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:43:52.867 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:43:52.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:52.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:53.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:54.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:54.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:54.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:43:54.020 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:43:54.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:54.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:54.116 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:54.108+0000 7f6b2e5188c0 -1 osd.0 30 log_to_monitors true 2026-02-17T05:43:54.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:43:55.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:43:55.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:43:55.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:43:55.170 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:43:55.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:43:55.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:43:55.320 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 34 up_thru 0 down_at 31 last_clean_interval [29,30) [v2:127.0.0.1:6802/3704065975,v1:127.0.0.1:6803/3704065975] [v2:127.0.0.1:6804/3704065975,v1:127.0.0.1:6805/3704065975] exists,up a8641ef1-a75d-413d-b693-c138b73925cc 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:43:55.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:43:55.322 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:43:55.322 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:43:55.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:43:55.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:43:55.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:43:55.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:43:55.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:43:55.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:43:55.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:43:55.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:43:55.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:43:55.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:43:55.579 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:43:55.579 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:43:55.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:43:55.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:55.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:43:55.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888066 2026-02-17T05:43:55.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888066 2026-02-17T05:43:55.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066' 2026-02-17T05:43:55.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:55.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:43:55.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705681 2026-02-17T05:43:55.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705681 2026-02-17T05:43:55.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705681' 2026-02-17T05:43:55.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:43:55.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:43:55.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542160 2026-02-17T05:43:55.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542160 2026-02-17T05:43:55.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705681 2-60129542160' 2026-02-17T05:43:55.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:55.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888066 2026-02-17T05:43:55.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:55.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:43:55.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888066 2026-02-17T05:43:55.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:55.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888066 2026-02-17T05:43:55.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888066' 2026-02-17T05:43:55.798 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 146028888066 2026-02-17T05:43:55.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:55.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888066 2026-02-17T05:43:55.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:43:56.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:43:56.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:43:57.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888066 -lt 146028888066 2026-02-17T05:43:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:57.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705681 2026-02-17T05:43:57.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:57.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:43:57.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705681 2026-02-17T05:43:57.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:57.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705681 2026-02-17T05:43:57.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705681' 2026-02-17T05:43:57.105 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705681 2026-02-17T05:43:57.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:43:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705682 -lt 38654705681 2026-02-17T05:43:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:43:57.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542160 2026-02-17T05:43:57.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:43:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:43:57.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542160 2026-02-17T05:43:57.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:43:57.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542160 2026-02-17T05:43:57.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542160' 2026-02-17T05:43:57.263 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542160 2026-02-17T05:43:57.264 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:43:57.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542160 -lt 60129542160 2026-02-17T05:43:57.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:43:57.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:57.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:57.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T05:43:57.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:43:57.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:43:57.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:43:57.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:43:57.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:43:57.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:43:57.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:43:57.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T05:43:57.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:43:57.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:43:57.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:43:57.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T05:43:57.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:43:57.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:43:57.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:432: TEST_auto_repair_bluestore_tag: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:43:57.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:433: TEST_auto_repair_bluestore_tag: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T05:43:57.934 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:43:35.628+0000 7fb93036d640 10 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T05:43:57.934 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:43:35.628+0000 7fb93036d640 10 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T05:43:57.934 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:43:35.628+0000 7fb93036d640 15 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T05:43:57.934 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:43:35.628+0000 7fb93036d640 20 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-17T05:43:57.934 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T05:43:35.628+0000 7fb93036d640 19 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T05:43:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:43:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:43:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:43:57.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:43:57.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:43:57.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:43:57.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:43:57.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:43:57.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:43:58.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:43:58.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:43:58.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:43:58.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:43:58.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:43:58.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:43:58.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:43:58.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:43:58.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:43:58.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:43:58.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:43:58.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:43:58.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:43:58.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:43:58.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:43:58.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:43:58.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:43:58.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:43:58.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:43:58.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:43:58.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:43:58.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:43:58.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:43:58.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:43:58.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:43:58.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:43:58.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:43:58.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:43:58.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:43:58.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:43:58.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:43:58.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:43:58.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:43:58.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:43:58.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:43:58.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:43:58.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:43:58.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:43:58.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:43:58.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:43:58.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:43:58.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:43:58.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:43:58.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:43:58.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:43:58.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_erasure_coded_appends td/osd-scrub-repair 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:309: TEST_auto_repair_erasure_coded_appends: auto_repair_erasure_coded td/osd-scrub-repair false 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: auto_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: auto_repair_erasure_coded: local allow_overwrites=false 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: auto_repair_erasure_coded: local poolname=ecpool 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: auto_repair_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:43:58.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:43:58.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:43:58.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:43:58.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:43:58.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:43:58.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:43:58.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:43:58.308 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:43:58.308 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.309 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:58.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:43:58.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:43:58.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:43:58.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:43:58.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:43:58.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:43:58.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:43:58.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:43:58.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:43:58.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:43:58.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:43:58.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:43:58.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:43:58.393 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:43:58.393 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:43:58.393 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:43:58.394 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:43:58.395 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.395 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.395 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:43:58.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:43:58.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:43:58.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:274: auto_repair_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T05:43:58.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:43:58.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:43:58.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:43:58.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:43:58.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:43:58.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:43:58.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:43:58.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:43:58.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:43:58.546 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:43:58.546 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.546 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:58.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:43:58.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:43:58.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:279: auto_repair_erasure_coded: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:43:58.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: seq 0 2 2026-02-17T05:43:58.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T05:43:58.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:43:58.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:43:58.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:43:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:43:58.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:43:58.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=acbff5f6-f561-4f7c-9264-b896a35b59f6 2026-02-17T05:43:58.572 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 acbff5f6-f561-4f7c-9264-b896a35b59f6 2026-02-17T05:43:58.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 acbff5f6-f561-4f7c-9264-b896a35b59f6' 2026-02-17T05:43:58.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:43:58.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAeAJRpCmswIhAAh4eXYTJ8TQqSJCkG+LI6VA== 2026-02-17T05:43:58.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAeAJRpCmswIhAAh4eXYTJ8TQqSJCkG+LI6VA=="}' 2026-02-17T05:43:58.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new acbff5f6-f561-4f7c-9264-b896a35b59f6 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:43:58.670 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:43:58.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:43:58.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAeAJRpCmswIhAAh4eXYTJ8TQqSJCkG+LI6VA== --osd-uuid acbff5f6-f561-4f7c-9264-b896a35b59f6 2026-02-17T05:43:58.695 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:58.684+0000 7faae5bbc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:58.697 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:58.688+0000 7faae5bbc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:58.698 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:58.688+0000 7faae5bbc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:43:58.698 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:58.688+0000 7faae5bbc8c0 -1 bdev(0x55976af73c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:43:58.699 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:43:58.688+0000 7faae5bbc8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:44:01.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:44:01.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:44:01.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:44:01.491 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:44:01.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:44:01.680 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:44:01.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:44:01.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:44:01.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:44:01.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:44:01.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:44:01.706 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:01.695+0000 7f1a834568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:01.707 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:01.695+0000 7f1a834568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:01.709 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:01.699+0000 7f1a834568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:01.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:44:01.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:44:01.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:44:01.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:44:01.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:44:01.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:01.835 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:44:01.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:44:01.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:01.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:44:01.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:44:02.993 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:44:02.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:02.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:44:03.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:03.193 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:03.183+0000 7f1a834568c0 -1 Falling back to public interface 2026-02-17T05:44:04.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:04.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:04.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:44:04.142 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:44:04.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:04.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:44:04.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:05.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:05.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:05.289 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:44:05.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:44:05.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:05.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:44:05.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:06.186 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:06.175+0000 7f1a834568c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:44:06.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:06.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:06.438 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:44:06.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:44:06.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:06.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:44:06.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:07.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:07.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:07.590 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:44:07.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:44:07.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:07.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:44:07.739 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1748830681,v1:127.0.0.1:6803/1748830681] [v2:127.0.0.1:6804/1748830681,v1:127.0.0.1:6805/1748830681] exists,up acbff5f6-f561-4f7c-9264-b896a35b59f6 2026-02-17T05:44:07.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:44:07.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:44:07.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:44:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:44:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=df5102d4-6638-44bc-a9fe-f0747e3aa038 2026-02-17T05:44:07.747 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 df5102d4-6638-44bc-a9fe-f0747e3aa038 2026-02-17T05:44:07.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 df5102d4-6638-44bc-a9fe-f0747e3aa038' 2026-02-17T05:44:07.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:44:07.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAnAJRpEuOCLBAAhz22TatymZMIXOpOcBPRxw== 2026-02-17T05:44:07.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAnAJRpEuOCLBAAhz22TatymZMIXOpOcBPRxw=="}' 2026-02-17T05:44:07.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new df5102d4-6638-44bc-a9fe-f0747e3aa038 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:44:07.906 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:44:07.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:44:07.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAnAJRpEuOCLBAAhz22TatymZMIXOpOcBPRxw== --osd-uuid df5102d4-6638-44bc-a9fe-f0747e3aa038 2026-02-17T05:44:07.934 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:07.923+0000 7f86df8898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:07.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:07.923+0000 7f86df8898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:07.937 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:07.927+0000 7f86df8898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:07.937 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:07.927+0000 7f86df8898c0 -1 bdev(0x55e064893c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:44:07.937 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:07.927+0000 7f86df8898c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:44:10.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:44:10.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:44:10.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:44:10.254 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:44:10.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:44:10.444 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:44:10.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:44:10.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:44:10.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:44:10.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:44:10.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:44:10.470 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:10.459+0000 7f5458b5c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:10.472 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:10.463+0000 7f5458b5c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:10.473 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:10.463+0000 7f5458b5c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:10.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:44:10.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:44:10.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:44:10.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:44:10.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:44:10.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:10.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:44:10.596 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:44:10.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:10.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:10.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:11.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:11.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:11.751 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:44:11.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:44:11.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:11.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:11.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:11.958 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:11.947+0000 7f5458b5c8c0 -1 Falling back to public interface 2026-02-17T05:44:12.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:12.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:12.906 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:44:12.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:44:12.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:12.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:14.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:14.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:14.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:44:14.060 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:44:14.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:14.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:14.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:14.945 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:14.935+0000 7f5458b5c8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:44:15.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:15.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:15.208 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:44:15.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:44:15.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:15.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:15.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:16.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:16.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:16.364 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:44:16.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:44:16.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:16.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:16.515 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/674535516,v1:127.0.0.1:6811/674535516] [v2:127.0.0.1:6812/674535516,v1:127.0.0.1:6813/674535516] exists,up df5102d4-6638-44bc-a9fe-f0747e3aa038 2026-02-17T05:44:16.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:44:16.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:44:16.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:44:16.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T05:44:16.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:44:16.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:44:16.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:44:16.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:44:16.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8f25949c-06f6-4274-a2e0-3ba3f10786a9 2026-02-17T05:44:16.523 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 8f25949c-06f6-4274-a2e0-3ba3f10786a9 2026-02-17T05:44:16.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 8f25949c-06f6-4274-a2e0-3ba3f10786a9' 2026-02-17T05:44:16.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:44:16.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAwAJRpgJtPHxAA+3o8y+SCmRiraXk3HUbW1Q== 2026-02-17T05:44:16.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAwAJRpgJtPHxAA+3o8y+SCmRiraXk3HUbW1Q=="}' 2026-02-17T05:44:16.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8f25949c-06f6-4274-a2e0-3ba3f10786a9 -i td/osd-scrub-repair/2/new.json 2026-02-17T05:44:16.684 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:44:16.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:44:16.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAwAJRpgJtPHxAA+3o8y+SCmRiraXk3HUbW1Q== --osd-uuid 8f25949c-06f6-4274-a2e0-3ba3f10786a9 2026-02-17T05:44:16.710 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:16.699+0000 7f21eeb478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:16.712 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:16.703+0000 7f21eeb478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:16.713 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:16.703+0000 7f21eeb478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:16.714 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:16.703+0000 7f21eeb478c0 -1 bdev(0x55dbef53bc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:44:16.714 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:16.703+0000 7f21eeb478c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:44:18.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:44:18.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:44:18.969 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:44:18.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:44:18.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:44:19.158 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:44:19.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:44:19.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:44:19.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:44:19.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:44:19.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:44:19.183 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:19.171+0000 7f71811448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:19.185 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:19.175+0000 7f71811448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:19.186 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:19.175+0000 7f71811448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:19.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:44:19.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:44:19.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:44:19.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:44:19.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:44:19.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:19.314 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:44:19.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:44:19.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:19.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:44:19.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:20.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:20.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:20.462 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:44:20.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:44:20.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:20.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:44:20.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:20.701 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:20.691+0000 7f71811448c0 -1 Falling back to public interface 2026-02-17T05:44:21.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:21.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:21.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:44:21.613 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:44:21.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:21.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:44:21.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:22.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:22.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:22.761 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:44:22.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:44:22.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:22.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:44:22.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:23.732 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:23.723+0000 7f71811448c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:44:23.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:23.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:23.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:44:23.916 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:44:23.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:23.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:44:24.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:25.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:25.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:25.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:44:25.072 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:44:25.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:25.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:44:25.220 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/273291154,v1:127.0.0.1:6819/273291154] [v2:127.0.0.1:6820/273291154,v1:127.0.0.1:6821/273291154] exists,up 8f25949c-06f6-4274-a2e0-3ba3f10786a9 2026-02-17T05:44:25.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:44:25.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:44:25.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:44:25.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:283: auto_repair_erasure_coded: create_rbd_pool 2026-02-17T05:44:25.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:44:25.363 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:44:25.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:44:25.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:44:25.569 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:44:25.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:44:26.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:44:26.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:284: auto_repair_erasure_coded: wait_for_clean 2026-02-17T05:44:26.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:44:26.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:44:26.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:44:26.862 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:44:26.862 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:44:26.862 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:44:26.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:44:26.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:44:26.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:44:26.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:44:26.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:44:26.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:44:26.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:44:26.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:44:26.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:44:27.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:44:27.117 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:44:27.117 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:44:27.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:44:27.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:27.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:44:27.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T05:44:27.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T05:44:27.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T05:44:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:27.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:44:27.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T05:44:27.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T05:44:27.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T05:44:27.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:27.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:44:27.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T05:44:27.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T05:44:27.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509442' 2026-02-17T05:44:27.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:27.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T05:44:27.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:27.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:44:27.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T05:44:27.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:27.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T05:44:27.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T05:44:27.336 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:44:27.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:27.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:44:27.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:44:28.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:44:28.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:28.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:44:28.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:44:29.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:44:29.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:29.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T05:44:29.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:29.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T05:44:29.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:29.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:44:29.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T05:44:29.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:29.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T05:44:29.791 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T05:44:29.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T05:44:29.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:44:29.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T05:44:29.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:29.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T05:44:29.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:29.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:44:29.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T05:44:29.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:29.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T05:44:29.956 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509442 2026-02-17T05:44:29.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T05:44:29.957 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:44:30.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509442 -lt 64424509442 2026-02-17T05:44:30.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:44:30.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:30.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:30.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:44:30.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:44:30.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:44:30.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:44:30.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:44:30.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:44:30.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:44:30.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:44:30.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:44:30.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:44:30.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:30.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:30.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:44:30.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:44:30.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:44:30.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: auto_repair_erasure_coded: create_ec_pool ecpool false k=2 m=1 2026-02-17T05:44:30.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T05:44:30.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T05:44:30.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T05:44:30.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T05:44:30.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T05:44:30.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T05:44:30.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T05:44:31.126 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T05:44:31.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:44:32.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T05:44:32.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T05:44:32.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:44:32.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:44:32.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:44:32.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:44:32.142 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:44:32.142 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:44:32.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:44:32.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:44:32.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:44:32.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:44:32.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:44:32.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:44:32.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:44:32.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:44:32.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:44:32.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:44:32.393 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:44:32.393 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:44:32.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:44:32.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:32.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:44:32.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T05:44:32.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T05:44:32.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T05:44:32.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:32.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:44:32.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T05:44:32.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T05:44:32.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966' 2026-02-17T05:44:32.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:32.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:44:32.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T05:44:32.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T05:44:32.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966 2-64424509444' 2026-02-17T05:44:32.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:32.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T05:44:32.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:32.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:44:32.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T05:44:32.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:32.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T05:44:32.613 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836488 2026-02-17T05:44:32.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T05:44:32.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:32.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-17T05:44:32.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:44:33.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:44:33.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:33.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T05:44:33.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:33.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T05:44:33.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:33.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:44:33.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T05:44:33.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:33.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T05:44:33.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T05:44:33.923 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672966 2026-02-17T05:44:33.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:44:34.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T05:44:34.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:34.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T05:44:34.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:34.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:44:34.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T05:44:34.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:34.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T05:44:34.087 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509444 2026-02-17T05:44:34.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T05:44:34.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:44:34.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T05:44:34.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:44:34.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:34.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:34.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:44:34.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:44:34.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:44:34.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:44:34.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:44:34.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:44:34.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:44:34.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:44:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:44:34.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:44:34.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:34.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:44:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:44:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:44:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T05:44:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:290: auto_repair_erasure_coded: local payload=ABCDEF 2026-02-17T05:44:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:291: auto_repair_erasure_coded: echo ABCDEF 2026-02-17T05:44:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:292: auto_repair_erasure_coded: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:44:34.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T05:44:34.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T05:44:34.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:44:34.797 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T05:44:34.797 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:44:34.797 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:44:34.798 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:44:34.798 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:44:34.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T05:44:34.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:44:34.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T05:44:35.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:44:35.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:44:35.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:44:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:44:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:44:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:44:36.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:44:36.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:44:36.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:44:36.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:44:36.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:44:36.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:44:36.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:44:36.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:44:37.053 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:44:37.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:44:37.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:44:37.591 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:44:37.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:44:37.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:44:37.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:44:37.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:44:37.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:44:37.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:44:37.618 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:37.607+0000 7f8c0a9f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:37.621 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:37.611+0000 7f8c0a9f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:37.622 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:37.611+0000 7f8c0a9f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:44:37.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:44:37.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:44:37.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:44:37.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:44:37.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:44:37.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:37.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:44:37.747 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:44:37.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:37.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:37.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:38.850 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:38.839+0000 7f8c0a9f18c0 -1 Falling back to public interface 2026-02-17T05:44:38.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:38.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:38.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:44:38.899 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:44:38.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:38.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:39.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:40.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:40.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:40.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:44:40.049 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:44:40.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:40.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:40.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:41.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:41.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:41.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:44:41.202 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:44:41.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:41.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:41.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:44:41.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:44:41.803+0000 7f8c0a9f18c0 -1 osd.1 28 log_to_monitors true 2026-02-17T05:44:42.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:44:42.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:44:42.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:44:42.355 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:44:42.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:44:42.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:44:42.504 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 32 up_thru 32 down_at 29 last_clean_interval [10,28) [v2:127.0.0.1:6810/2429126059,v1:127.0.0.1:6811/2429126059] [v2:127.0.0.1:6812/2429126059,v1:127.0.0.1:6813/2429126059] exists,up df5102d4-6638-44bc-a9fe-f0747e3aa038 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:44:42.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:44:42.506 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:44:42.507 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:44:42.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:44:42.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:44:42.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:44:42.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:44:42.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:44:42.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:44:42.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:44:42.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:44:42.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:44:42.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:44:42.764 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:44:42.764 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:44:42.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:44:42.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:42.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:44:42.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T05:44:42.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T05:44:42.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T05:44:42.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:42.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:44:42.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953474 2026-02-17T05:44:42.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953474 2026-02-17T05:44:42.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-137438953474' 2026-02-17T05:44:42.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:42.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:44:42.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509447 2026-02-17T05:44:42.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509447 2026-02-17T05:44:42.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-137438953474 2-64424509447' 2026-02-17T05:44:42.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:42.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T05:44:42.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:42.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:44:42.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T05:44:42.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:42.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T05:44:42.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T05:44:42.981 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836491 2026-02-17T05:44:42.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:43.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T05:44:43.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:44:44.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:44:44.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:44.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T05:44:44.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:44:45.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:44:45.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:45.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T05:44:45.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:45.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-137438953474 2026-02-17T05:44:45.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:45.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:44:45.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-137438953474 2026-02-17T05:44:45.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:45.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953474 2026-02-17T05:44:45.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 137438953474' 2026-02-17T05:44:45.451 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 137438953474 2026-02-17T05:44:45.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:44:45.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953474 -lt 137438953474 2026-02-17T05:44:45.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:45.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509447 2026-02-17T05:44:45.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:45.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:44:45.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509447 2026-02-17T05:44:45.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:45.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509447 2026-02-17T05:44:45.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-17T05:44:45.619 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509447 2026-02-17T05:44:45.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:44:45.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509447 -lt 64424509447 2026-02-17T05:44:45.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:44:45.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:45.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:44:45.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:44:46.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:44:46.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:44:46.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:46.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:46.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:44:46.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:44:46.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:44:46.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T05:44:46.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:44:46.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:44:46.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:44:46.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:44:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: local pgid=2.0 2026-02-17T05:44:46.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: get_last_scrub_stamp 2.0 2026-02-17T05:44:46.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:46.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:46.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:46.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:46.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: wait_for_scrub 2.0 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:46.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:46.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:46.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:46.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:47.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:47.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:47.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:47.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:47.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:47.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:47.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:47.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:47.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:48.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:48.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:48.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:48.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:48.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:48.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:48.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:49.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:49.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:50.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:50.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:50.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:50.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:50.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:50.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:50.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:50.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:50.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:51.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:51.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:51.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:51.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:51.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:51.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:51.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:51.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:51.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:52.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:52.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:52.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:52.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:52.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:52.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:52.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:52.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:52.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:53.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:53.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:53.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:53.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:53.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:53.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:53.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:53.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:53.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:54.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:54.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:54.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:54.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:54.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:54.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:54.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:54.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:54.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:55.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:55.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:55.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:55.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:55.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:55.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:55.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:55.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:42.671111+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:55.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:44:56.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:44:56.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:44:56.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:44:56.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:44:56.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:44:56.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:44:56.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:44:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:44:51.581967+0000 '>' 2026-02-17T05:44:42.671111+0000 2026-02-17T05:44:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:44:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:300: auto_repair_erasure_coded: wait_for_clean 2026-02-17T05:44:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:44:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:44:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:44:57.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:44:57.102 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:44:57.102 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:44:57.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:44:57.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:44:57.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:44:57.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:44:57.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:44:57.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:44:57.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:44:57.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:44:57.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:44:57.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:44:57.366 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:44:57.366 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:44:57.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:44:57.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:57.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:44:57.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836495 2026-02-17T05:44:57.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836495 2026-02-17T05:44:57.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495' 2026-02-17T05:44:57.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:57.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:44:57.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953478 2026-02-17T05:44:57.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953478 2026-02-17T05:44:57.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-137438953478' 2026-02-17T05:44:57.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:44:57.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:44:57.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509451 2026-02-17T05:44:57.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509451 2026-02-17T05:44:57.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-137438953478 2-64424509451' 2026-02-17T05:44:57.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:57.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836495 2026-02-17T05:44:57.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:57.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:44:57.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836495 2026-02-17T05:44:57.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:57.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836495 2026-02-17T05:44:57.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-17T05:44:57.588 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836495 2026-02-17T05:44:57.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:57.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-17T05:44:57.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:44:58.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:44:58.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:44:58.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-17T05:44:58.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:58.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-137438953478 2026-02-17T05:44:58.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:58.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:44:58.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-137438953478 2026-02-17T05:44:58.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:58.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953478 2026-02-17T05:44:58.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 137438953478' 2026-02-17T05:44:58.907 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 137438953478 2026-02-17T05:44:58.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:44:59.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953478 -lt 137438953478 2026-02-17T05:44:59.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:44:59.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509451 2026-02-17T05:44:59.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:44:59.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:44:59.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509451 2026-02-17T05:44:59.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:44:59.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509451 2026-02-17T05:44:59.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509451' 2026-02-17T05:44:59.071 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509451 2026-02-17T05:44:59.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:44:59.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509451 -lt 64424509451 2026-02-17T05:44:59.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:44:59.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:59.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:59.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:44:59.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:44:59.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:44:59.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:44:59.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:44:59.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:44:59.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:44:59.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:44:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:44:59.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:44:59.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:44:59.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:44:59.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:44:59.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:44:59.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:44:59.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T05:44:59.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T05:44:59.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:44:59.741 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T05:44:59.741 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:44:59.741 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:44:59.741 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:44:59.742 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:44:59.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T05:44:59.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:44:59.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T05:45:00.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:45:00.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:45:00.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:45:00.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:45:00.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:45:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:45:01.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:45:01.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:45:01.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:45:01.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:45:01.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:45:01.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:45:01.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:45:01.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:45:01.681 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T05:45:01.682 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-17T05:45:01.682 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:45:01.682 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:45:01.682 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:45:02.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:02.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:45:02.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:02.218 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:45:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:45:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:45:02.220 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:45:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:02.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:45:02.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:45:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:45:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:45:02.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:45:02.240 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:02.231+0000 7fcf9c3f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:02.243 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:02.231+0000 7fcf9c3f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:02.244 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:02.235+0000 7fcf9c3f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:45:02.375 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:45:02.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:02.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:02.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:03.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:03.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:03.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:45:03.526 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:45:03.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:03.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:03.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:03.746 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:03.734+0000 7fcf9c3f48c0 -1 Falling back to public interface 2026-02-17T05:45:04.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:04.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:04.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:45:04.676 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:45:04.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:04.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:04.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:05.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:05.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:05.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:45:05.823 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:45:05.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:05.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:05.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:06.724 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:06.710+0000 7fcf9c3f48c0 -1 osd.1 35 log_to_monitors true 2026-02-17T05:45:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:45:06.973 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:45:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:07.118 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [32,35) [v2:127.0.0.1:6810/336881538,v1:127.0.0.1:6811/336881538] [v2:127.0.0.1:6812/336881538,v1:127.0.0.1:6813/336881538] exists,up df5102d4-6638-44bc-a9fe-f0747e3aa038 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:45:07.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:45:07.120 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:45:07.120 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:45:07.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:45:07.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:45:07.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:45:07.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:45:07.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:45:07.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:45:07.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:45:07.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:45:07.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:45:07.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:45:07.377 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:45:07.377 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:45:07.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:45:07.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:07.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:45:07.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T05:45:07.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T05:45:07.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T05:45:07.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:07.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:45:07.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724546 2026-02-17T05:45:07.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724546 2026-02-17T05:45:07.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-167503724546' 2026-02-17T05:45:07.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:07.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:45:07.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509454 2026-02-17T05:45:07.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509454 2026-02-17T05:45:07.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-167503724546 2-64424509454' 2026-02-17T05:45:07.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:07.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T05:45:07.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:07.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:45:07.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T05:45:07.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:07.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T05:45:07.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T05:45:07.597 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836498 2026-02-17T05:45:07.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:07.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836498 2026-02-17T05:45:07.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:45:08.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:45:08.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:08.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-17T05:45:08.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:08.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-167503724546 2026-02-17T05:45:08.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:08.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:45:08.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-167503724546 2026-02-17T05:45:08.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:08.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724546 2026-02-17T05:45:08.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 167503724546' 2026-02-17T05:45:08.911 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 167503724546 2026-02-17T05:45:08.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:45:09.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-17T05:45:09.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:09.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509454 2026-02-17T05:45:09.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:09.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:45:09.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509454 2026-02-17T05:45:09.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:09.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509454 2026-02-17T05:45:09.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2026-02-17T05:45:09.074 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509454 2026-02-17T05:45:09.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:45:09.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509454 -lt 64424509454 2026-02-17T05:45:09.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:45:09.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:09.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:09.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:45:09.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:45:09.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:45:09.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:45:09.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:45:09.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:45:09.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:45:09.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:45:09.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:45:09.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:45:09.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:09.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:09.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:45:09.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:45:09.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:45:09.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: auto_repair_erasure_coded: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:45:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:305: auto_repair_erasure_coded: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:45:09.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:45:09.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:45:09.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:45:09.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:45:09.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:45:09.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:45:09.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:45:09.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:45:09.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:45:09.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:45:09.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:45:09.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:45:09.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:45:09.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:45:09.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:45:09.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:45:09.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:45:09.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:45:09.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:45:09.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:45:09.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:45:09.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:45:09.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:45:09.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:45:09.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:09.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:09.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:45:09.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:45:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:45:09.895 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:45:09.895 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:45:09.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:45:09.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:45:09.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:45:09.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:45:09.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:45:09.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:45:09.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:45:09.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:45:09.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:45:09.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:45:09.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:45:09.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:45:09.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:45:09.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:45:09.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:45:09.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:45:09.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:45:09.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:45:09.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:09.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:09.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:45:09.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:45:09.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:45:09.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:45:09.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:45:09.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:09.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:09.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:45:09.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_erasure_coded_overwrites td/osd-scrub-repair 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: TEST_auto_repair_erasure_coded_overwrites: '[' true = true ']' 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: TEST_auto_repair_erasure_coded_overwrites: auto_repair_erasure_coded td/osd-scrub-repair true 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: auto_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: auto_repair_erasure_coded: local allow_overwrites=true 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: auto_repair_erasure_coded: local poolname=ecpool 2026-02-17T05:45:09.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: auto_repair_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T05:45:09.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:45:09.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:45:09.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:45:09.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:45:09.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:45:09.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:45:09.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:45:09.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:45:09.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:45:09.932 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:45:09.932 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:09.932 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:09.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:09.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:45:09.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:45:09.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:45:09.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:45:09.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:45:09.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:45:09.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:45:09.961 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:45:09.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:45:10.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:45:10.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:45:10.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:45:10.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:45:10.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:45:10.032 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:45:10.032 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:45:10.032 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:45:10.033 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:45:10.033 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:10.033 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:10.033 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:45:10.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:45:10.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:45:10.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:274: auto_repair_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T05:45:10.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:45:10.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:45:10.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:45:10.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:45:10.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:45:10.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:45:10.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:45:10.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:45:10.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:45:10.183 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:45:10.183 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:10.183 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:10.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:10.184 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:45:10.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:45:10.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:279: auto_repair_erasure_coded: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:45:10.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: seq 0 2 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:45:10.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:45:10.206 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:45:10.206 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:10.206 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:10.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:10.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:10.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:45:10.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:45:10.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:45:10.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4ae86250-8f23-4942-8d7d-7224585c71fc 2026-02-17T05:45:10.208 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 4ae86250-8f23-4942-8d7d-7224585c71fc 2026-02-17T05:45:10.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 4ae86250-8f23-4942-8d7d-7224585c71fc' 2026-02-17T05:45:10.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:45:10.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBmAJRp6MGKDBAAxadfovWCW05C3PH5V4vO8Q== 2026-02-17T05:45:10.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBmAJRp6MGKDBAAxadfovWCW05C3PH5V4vO8Q=="}' 2026-02-17T05:45:10.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4ae86250-8f23-4942-8d7d-7224585c71fc -i td/osd-scrub-repair/0/new.json 2026-02-17T05:45:10.310 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:45:10.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:45:10.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBmAJRp6MGKDBAAxadfovWCW05C3PH5V4vO8Q== --osd-uuid 4ae86250-8f23-4942-8d7d-7224585c71fc 2026-02-17T05:45:10.335 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:10.322+0000 7f7294b0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:10.337 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:10.326+0000 7f7294b0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:10.338 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:10.326+0000 7f7294b0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:10.339 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:10.326+0000 7f7294b0e8c0 -1 bdev(0x55e5b26b1c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:10.339 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:10.326+0000 7f7294b0e8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:45:12.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:45:12.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:45:12.959 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:45:12.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:45:12.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:45:13.149 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:45:13.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:45:13.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:13.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:45:13.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:45:13.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:45:13.174 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:13.162+0000 7f8cd18fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:13.175 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:13.162+0000 7f8cd18fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:13.176 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:13.166+0000 7f8cd18fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:13.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:45:13.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:45:13.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:45:13.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:45:13.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:45:13.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:13.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:45:13.304 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:45:13.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:13.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:45:13.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:14.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:14.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:14.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:45:14.456 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:45:14.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:14.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:45:14.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:14.682 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:14.670+0000 7f8cd18fc8c0 -1 Falling back to public interface 2026-02-17T05:45:15.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:15.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:15.607 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:45:15.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:45:15.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:15.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:45:15.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:16.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:16.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:16.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:45:16.758 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:45:16.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:16.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:45:16.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:17.642 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:17.630+0000 7f8cd18fc8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:45:17.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:17.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:17.906 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:45:17.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:45:17.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:17.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:45:18.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:19.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:19.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:19.055 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:45:19.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:45:19.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:19.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:45:19.207 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2605402641,v1:127.0.0.1:6803/2605402641] [v2:127.0.0.1:6804/2605402641,v1:127.0.0.1:6805/2605402641] exists,up 4ae86250-8f23-4942-8d7d-7224585c71fc 2026-02-17T05:45:19.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:45:19.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:45:19.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:45:19.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T05:45:19.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:45:19.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:45:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:45:19.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:45:19.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d53b2250-5c6a-4a51-b083-176f5405e889 2026-02-17T05:45:19.215 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 d53b2250-5c6a-4a51-b083-176f5405e889 2026-02-17T05:45:19.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 d53b2250-5c6a-4a51-b083-176f5405e889' 2026-02-17T05:45:19.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:45:19.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBvAJRpZ2nXDBAA3dhenzzlkj6kgptSGwcXHg== 2026-02-17T05:45:19.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBvAJRpZ2nXDBAA3dhenzzlkj6kgptSGwcXHg=="}' 2026-02-17T05:45:19.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d53b2250-5c6a-4a51-b083-176f5405e889 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:45:19.377 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:45:19.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:45:19.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBvAJRpZ2nXDBAA3dhenzzlkj6kgptSGwcXHg== --osd-uuid d53b2250-5c6a-4a51-b083-176f5405e889 2026-02-17T05:45:19.404 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:19.390+0000 7f4e69c9b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:19.405 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:19.394+0000 7f4e69c9b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:19.406 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:19.394+0000 7f4e69c9b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:19.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:19.394+0000 7f4e69c9b8c0 -1 bdev(0x560f8bf5dc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:19.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:19.394+0000 7f4e69c9b8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:45:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:45:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:45:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:45:21.679 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:45:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:45:21.869 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:45:21.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:45:21.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:21.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:45:21.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:45:21.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:45:21.895 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:21.882+0000 7f4f742508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:21.896 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:21.886+0000 7f4f742508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:21.898 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:21.886+0000 7f4f742508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:22.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:23.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:23.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:23.172 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:45:23.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:45:23.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:23.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:23.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:23.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:23.402+0000 7f4f742508c0 -1 Falling back to public interface 2026-02-17T05:45:24.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:24.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:24.325 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:45:24.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:45:24.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:24.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:24.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:25.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:25.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:25.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:45:25.472 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:45:25.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:25.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:25.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:26.419 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:26.406+0000 7f4f742508c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:45:26.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:26.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:26.629 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:45:26.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:45:26.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:26.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:26.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:27.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:27.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:27.806 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:45:27.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:45:27.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:27.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:27.951 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/387411868,v1:127.0.0.1:6811/387411868] [v2:127.0.0.1:6812/387411868,v1:127.0.0.1:6813/387411868] exists,up d53b2250-5c6a-4a51-b083-176f5405e889 2026-02-17T05:45:27.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:45:27.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:45:27.956 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:45:27.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:45:27.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:45:27.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:45:27.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:45:27.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:45:27.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:45:27.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=88a57494-4dc2-4fac-9cc5-5eb1a198038b 2026-02-17T05:45:27.959 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 88a57494-4dc2-4fac-9cc5-5eb1a198038b 2026-02-17T05:45:27.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 88a57494-4dc2-4fac-9cc5-5eb1a198038b' 2026-02-17T05:45:27.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:45:27.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB3AJRpMu9DORAABbgTLcUtD8mMOuQDc1pO/A== 2026-02-17T05:45:27.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB3AJRpMu9DORAABbgTLcUtD8mMOuQDc1pO/A=="}' 2026-02-17T05:45:27.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 88a57494-4dc2-4fac-9cc5-5eb1a198038b -i td/osd-scrub-repair/2/new.json 2026-02-17T05:45:28.113 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:45:28.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:45:28.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQB3AJRpMu9DORAABbgTLcUtD8mMOuQDc1pO/A== --osd-uuid 88a57494-4dc2-4fac-9cc5-5eb1a198038b 2026-02-17T05:45:28.149 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:28.138+0000 7f374de368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:28.151 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:28.138+0000 7f374de368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:28.152 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:28.138+0000 7f374de368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:28.152 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:28.142+0000 7f374de368c0 -1 bdev(0x561987c1fc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:45:28.153 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:28.142+0000 7f374de368c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:45:30.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:45:30.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:45:30.436 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:45:30.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:45:30.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:45:30.622 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:45:30.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:45:30.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:30.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:45:30.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:45:30.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:45:30.647 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:30.634+0000 7fb7c8b918c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:30.649 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:30.638+0000 7fb7c8b918c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:30.650 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:30.638+0000 7fb7c8b918c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:30.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:45:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:45:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:45:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:45:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:45:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:30.777 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:45:30.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:45:30.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:30.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:45:30.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:31.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:31.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:31.927 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:45:31.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:45:31.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:31.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:45:32.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:32.130 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:32.118+0000 7fb7c8b918c0 -1 Falling back to public interface 2026-02-17T05:45:33.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:33.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:33.074 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:45:33.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:45:33.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:33.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:45:33.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:34.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:34.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:34.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:45:34.224 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:45:34.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:34.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:45:34.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:35.121 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:35.106+0000 7fb7c8b918c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:45:35.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:35.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:35.370 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:45:35.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:45:35.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:35.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:45:35.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:36.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:36.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:36.521 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:45:36.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:45:36.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:45:36.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:36.668 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1184416902,v1:127.0.0.1:6819/1184416902] [v2:127.0.0.1:6820/1184416902,v1:127.0.0.1:6821/1184416902] exists,up 88a57494-4dc2-4fac-9cc5-5eb1a198038b 2026-02-17T05:45:36.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:45:36.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:45:36.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:45:36.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:283: auto_repair_erasure_coded: create_rbd_pool 2026-02-17T05:45:36.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:45:36.807 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:45:36.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:45:36.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:45:37.027 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:45:37.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:45:38.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:45:38.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:284: auto_repair_erasure_coded: wait_for_clean 2026-02-17T05:45:38.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:45:38.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:45:38.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:45:38.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:45:38.319 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:45:38.319 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:45:38.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:45:38.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:45:38.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:45:38.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:45:38.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:45:38.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:45:38.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:45:38.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:45:38.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:45:38.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:45:38.576 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:45:38.576 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:45:38.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:45:38.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:38.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:45:38.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T05:45:38.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T05:45:38.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T05:45:38.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:38.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:45:38.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T05:45:38.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T05:45:38.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T05:45:38.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:38.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:45:38.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T05:45:38.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T05:45:38.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509442' 2026-02-17T05:45:38.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:38.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T05:45:38.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:38.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:45:38.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T05:45:38.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:38.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T05:45:38.794 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T05:45:38.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T05:45:38.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:38.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:45:38.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:45:39.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:45:39.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:40.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T05:45:40.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:45:41.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:45:41.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:41.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T05:45:41.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:41.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T05:45:41.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:41.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:45:41.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T05:45:41.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:41.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T05:45:41.269 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T05:45:41.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T05:45:41.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:45:41.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T05:45:41.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:41.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T05:45:41.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:41.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:45:41.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T05:45:41.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:41.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T05:45:41.425 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509442 2026-02-17T05:45:41.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T05:45:41.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:45:41.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509442 -lt 64424509442 2026-02-17T05:45:41.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:45:41.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:41.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:41.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:45:41.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:45:41.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:45:41.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:45:41.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:45:41.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:45:41.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:45:41.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:45:41.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:45:41.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:45:41.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:41.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:45:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:45:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:45:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: auto_repair_erasure_coded: create_ec_pool ecpool true k=2 m=1 2026-02-17T05:45:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T05:45:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T05:45:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T05:45:42.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T05:45:42.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T05:45:42.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T05:45:42.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T05:45:42.629 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T05:45:42.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:45:43.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T05:45:43.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T05:45:43.833 INFO:tasks.workunit.client.0.trial074.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T05:45:43.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T05:45:43.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:45:43.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:45:43.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:45:43.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:45:43.845 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:45:43.845 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:45:43.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:45:43.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:45:43.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:45:43.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:45:43.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:45:43.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:45:43.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:45:43.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:45:43.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:45:44.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:45:44.108 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:45:44.109 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:45:44.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:45:44.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:44.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:45:44.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T05:45:44.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T05:45:44.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T05:45:44.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:44.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:45:44.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T05:45:44.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T05:45:44.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966' 2026-02-17T05:45:44.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:44.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:45:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T05:45:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T05:45:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966 2-64424509444' 2026-02-17T05:45:44.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:44.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T05:45:44.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:44.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:45:44.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T05:45:44.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T05:45:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T05:45:44.325 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836488 2026-02-17T05:45:44.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:44.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-17T05:45:44.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:45:45.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:45:45.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:45.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T05:45:45.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:45.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T05:45:45.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:45.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:45:45.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T05:45:45.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:45.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T05:45:45.635 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672966 2026-02-17T05:45:45.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T05:45:45.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:45:45.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T05:45:45.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:45.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T05:45:45.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:45.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:45:45.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T05:45:45.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:45.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T05:45:45.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T05:45:45.799 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509444 2026-02-17T05:45:45.799 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:45:45.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T05:45:45.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:45:45.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:45.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:46.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:45:46.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:45:46.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:45:46.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:45:46.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:45:46.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:45:46.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:45:46.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:45:46.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:45:46.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:45:46.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:46.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:45:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:45:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:45:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T05:45:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:290: auto_repair_erasure_coded: local payload=ABCDEF 2026-02-17T05:45:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:291: auto_repair_erasure_coded: echo ABCDEF 2026-02-17T05:45:46.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:292: auto_repair_erasure_coded: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:45:46.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T05:45:46.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T05:45:46.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:45:46.499 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T05:45:46.499 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:45:46.499 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:45:46.499 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:45:46.499 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:45:46.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T05:45:46.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:45:46.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T05:45:46.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:45:46.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:45:46.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:45:46.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:45:48.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:45:48.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:45:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:45:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:45:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:45:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:45:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:45:48.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:45:48.750 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:45:49.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:49.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:45:49.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:45:49.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:45:49.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:49.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:45:49.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:45:49.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:45:49.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:45:49.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:45:49.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:45:49.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:45:49.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:45:49.327 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:49.314+0000 7fdb57cf38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:49.328 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:49.314+0000 7fdb57cf38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:49.329 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:49.318+0000 7fdb57cf38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:45:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:45:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:45:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:45:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:45:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:45:49.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:49.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:45:49.453 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:45:49.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:49.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:49.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:50.582 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:50.570+0000 7fdb57cf38c0 -1 Falling back to public interface 2026-02-17T05:45:50.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:50.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:50.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:45:50.603 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:45:50.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:50.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:50.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:51.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:51.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:51.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:45:51.752 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:45:51.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:51.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:51.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:52.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:52.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:52.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:45:52.905 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:45:52.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:52.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:53.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:45:53.550 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:45:53.538+0000 7fdb57cf38c0 -1 osd.1 30 log_to_monitors true 2026-02-17T05:45:54.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:45:54.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:45:54.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:45:54.061 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:45:54.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:45:54.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:45:54.211 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 34 up_thru 34 down_at 31 last_clean_interval [10,30) [v2:127.0.0.1:6810/2787545966,v1:127.0.0.1:6811/2787545966] [v2:127.0.0.1:6812/2787545966,v1:127.0.0.1:6813/2787545966] exists,up d53b2250-5c6a-4a51-b083-176f5405e889 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:45:54.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:45:54.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:45:54.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:45:54.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:45:54.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:45:54.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:45:54.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:45:54.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:45:54.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:45:54.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:45:54.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:45:54.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:45:54.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:45:54.479 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:45:54.479 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:45:54.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:45:54.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:54.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:45:54.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T05:45:54.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T05:45:54.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T05:45:54.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:54.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:45:54.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888066 2026-02-17T05:45:54.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888066 2026-02-17T05:45:54.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-146028888066' 2026-02-17T05:45:54.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:45:54.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:45:54.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509447 2026-02-17T05:45:54.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509447 2026-02-17T05:45:54.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-146028888066 2-64424509447' 2026-02-17T05:45:54.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:54.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T05:45:54.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:54.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:45:54.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T05:45:54.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:54.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T05:45:54.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T05:45:54.696 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836491 2026-02-17T05:45:54.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:54.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T05:45:54.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:45:55.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:45:55.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:56.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T05:45:56.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:45:57.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:45:57.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:45:57.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T05:45:57.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:57.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888066 2026-02-17T05:45:57.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:57.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:45:57.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888066 2026-02-17T05:45:57.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:57.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888066 2026-02-17T05:45:57.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888066' 2026-02-17T05:45:57.164 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 146028888066 2026-02-17T05:45:57.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:45:57.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888066 -lt 146028888066 2026-02-17T05:45:57.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:45:57.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509447 2026-02-17T05:45:57.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:45:57.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:45:57.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509447 2026-02-17T05:45:57.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:45:57.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509447 2026-02-17T05:45:57.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-17T05:45:57.333 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509447 2026-02-17T05:45:57.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:45:57.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509448 -lt 64424509447 2026-02-17T05:45:57.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:45:57.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:57.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:57.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:45:57.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:45:57.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:45:57.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:45:57.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:45:57.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:45:57.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:45:57.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:45:57.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:45:57.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:45:57.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:45:57.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:45:57.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:45:57.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:45:57.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:45:57.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T05:45:57.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:45:57.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:45:57.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:45:57.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:45:58.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: local pgid=2.0 2026-02-17T05:45:58.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: get_last_scrub_stamp 2.0 2026-02-17T05:45:58.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:45:58.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:45:58.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:45:58.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:45:58.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: wait_for_scrub 2.0 2026-02-17T05:45:54.131028+0000 2026-02-17T05:45:58.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:45:58.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:45:54.131028+0000 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:45:58.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:45:58.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:45:54.131028+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:45:58.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:45:59.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:45:59.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:45:59.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:45:59.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:45:59.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:45:59.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:45:59.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:45:59.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:45:54.131028+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:45:59.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:46:00.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:46:00.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:46:00.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:46:00.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:46:00.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:46:00.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:46:00.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:46:00.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:45:54.131028+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:46:00.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:46:01.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:46:01.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:46:01.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:46:01.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:46:01.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:46:01.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:46:01.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:46:01.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:45:54.131028+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:46:01.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:46:02.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:46:02.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:46:02.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:46:02.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:46:02.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:46:02.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:46:02.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:46:03.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:45:54.131028+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:46:03.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:46:04.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:46:04.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:46:04.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:46:04.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:46:04.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:46:04.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:46:04.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:46:04.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:45:54.131028+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:46:04.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:46:05.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:46:05.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:46:05.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:46:05.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:46:05.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:46:05.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:46:05.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:46:05.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:45:54.131028+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:46:05.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:46:06.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:46:06.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:46:06.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:46:06.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:46:06.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:46:06.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:46:06.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:46:06.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:46:04.135946+0000 '>' 2026-02-17T05:45:54.131028+0000 2026-02-17T05:46:06.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:46:06.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:300: auto_repair_erasure_coded: wait_for_clean 2026-02-17T05:46:06.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:46:06.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:46:06.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:46:06.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:46:06.528 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:46:06.528 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:46:06.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:46:06.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:46:06.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:46:06.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:46:06.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:46:06.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:46:06.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:46:06.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:46:06.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:46:06.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:46:06.780 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:46:06.780 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:46:06.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:46:06.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:06.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:46:06.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836494 2026-02-17T05:46:06.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836494 2026-02-17T05:46:06.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494' 2026-02-17T05:46:06.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:06.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:46:06.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888069 2026-02-17T05:46:06.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888069 2026-02-17T05:46:06.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-146028888069' 2026-02-17T05:46:06.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:06.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:46:06.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509451 2026-02-17T05:46:06.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509451 2026-02-17T05:46:06.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-146028888069 2-64424509451' 2026-02-17T05:46:06.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:06.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836494 2026-02-17T05:46:06.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:06.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:46:06.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836494 2026-02-17T05:46:06.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:06.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836494 2026-02-17T05:46:06.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-17T05:46:06.999 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836494 2026-02-17T05:46:07.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:46:07.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T05:46:07.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:46:08.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:46:08.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:46:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T05:46:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:46:09.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:46:09.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:46:09.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 21474836494 2026-02-17T05:46:09.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:09.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888069 2026-02-17T05:46:09.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:09.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:46:09.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888069 2026-02-17T05:46:09.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:09.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888069 2026-02-17T05:46:09.468 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 146028888069 2026-02-17T05:46:09.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888069' 2026-02-17T05:46:09.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:46:09.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888069 -lt 146028888069 2026-02-17T05:46:09.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:09.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509451 2026-02-17T05:46:09.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:09.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:46:09.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509451 2026-02-17T05:46:09.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:09.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509451 2026-02-17T05:46:09.628 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509451 2026-02-17T05:46:09.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509451' 2026-02-17T05:46:09.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:46:09.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509451 -lt 64424509451 2026-02-17T05:46:09.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:46:09.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:46:09.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:46:09.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:46:09.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:46:09.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:46:09.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:46:09.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:46:09.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:46:09.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:46:09.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:46:10.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:46:10.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:46:10.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:46:10.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:46:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:46:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:46:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:46:10.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T05:46:10.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T05:46:10.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:46:10.309 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T05:46:10.309 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:46:10.309 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:46:10.309 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:46:10.309 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:46:10.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T05:46:10.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:46:10.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T05:46:10.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:46:10.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:46:10.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:46:10.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:46:10.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:46:10.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:46:10.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:46:10.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:46:11.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:46:11.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:46:11.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:46:11.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:46:11.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:46:11.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:46:11.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:46:11.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:46:12.242 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T05:46:12.242 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-17T05:46:12.243 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:46:12.243 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:46:12.243 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:46:12.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:46:12.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:46:12.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:46:12.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:46:12.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:46:12.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:12.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:12.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T05:46:12.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:46:12.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:46:12.785 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:46:12.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T05:46:12.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:46:12.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:46:12.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:46:12.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:46:12.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:46:12.810 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:12.797+0000 7ff06a4de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:12.811 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:12.797+0000 7ff06a4de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:12.812 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:12.797+0000 7ff06a4de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:12.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:46:12.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:46:12.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:46:12.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:46:12.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:46:12.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:12.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:46:12.936 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:46:12.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:12.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:13.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:14.071 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:14.057+0000 7ff06a4de8c0 -1 Falling back to public interface 2026-02-17T05:46:14.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:14.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:14.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:46:14.088 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:46:14.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:14.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:14.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:15.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:15.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:15.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:46:15.240 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:46:15.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:15.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:15.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:16.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:16.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:16.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:46:16.390 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:46:16.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:16.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:16.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:17.070 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:17.057+0000 7ff06a4de8c0 -1 osd.1 36 log_to_monitors true 2026-02-17T05:46:17.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:17.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:17.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:46:17.540 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:46:17.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:17.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:17.691 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [34,36) [v2:127.0.0.1:6810/1966356247,v1:127.0.0.1:6811/1966356247] [v2:127.0.0.1:6812/1966356247,v1:127.0.0.1:6813/1966356247] exists,up d53b2250-5c6a-4a51-b083-176f5405e889 2026-02-17T05:46:17.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:46:17.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:46:17.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:46:17.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:46:17.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:46:17.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:46:17.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:46:17.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:46:17.693 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:46:17.693 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:46:17.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:46:17.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:46:17.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:46:17.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:46:17.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:46:17.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:46:17.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:46:17.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:46:17.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:46:17.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:46:17.943 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:46:17.943 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T05:46:17.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:46:17.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:17.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:46:18.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T05:46:18.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T05:46:18.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T05:46:18.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:18.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:46:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691842 2026-02-17T05:46:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691842 2026-02-17T05:46:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-171798691842' 2026-02-17T05:46:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:18.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:46:18.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509454 2026-02-17T05:46:18.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509454 2026-02-17T05:46:18.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-171798691842 2-64424509454' 2026-02-17T05:46:18.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:18.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T05:46:18.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:18.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:46:18.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T05:46:18.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:18.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T05:46:18.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T05:46:18.157 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836497 2026-02-17T05:46:18.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:46:18.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T05:46:18.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:46:19.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:46:19.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:46:19.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-17T05:46:19.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:19.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691842 2026-02-17T05:46:19.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:19.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:46:19.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691842 2026-02-17T05:46:19.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:19.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691842 2026-02-17T05:46:19.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691842' 2026-02-17T05:46:19.468 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 171798691842 2026-02-17T05:46:19.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:46:19.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-17T05:46:19.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:19.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509454 2026-02-17T05:46:19.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:19.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:46:19.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509454 2026-02-17T05:46:19.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:19.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509454 2026-02-17T05:46:19.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2026-02-17T05:46:19.633 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509454 2026-02-17T05:46:19.634 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:46:19.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509454 -lt 64424509454 2026-02-17T05:46:19.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:46:19.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:46:19.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:46:19.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:46:19.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:46:19.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:46:19.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:46:19.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:46:19.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:46:19.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:46:19.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:46:20.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:46:20.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:46:20.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:46:20.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:46:20.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:46:20.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:46:20.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:46:20.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: auto_repair_erasure_coded: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:46:20.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:305: auto_repair_erasure_coded: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:46:20.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:46:20.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:46:20.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:46:20.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:46:20.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:46:20.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:46:20.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:46:20.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:46:20.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:46:20.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:46:20.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:46:20.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:46:20.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:46:20.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:46:20.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:46:20.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:46:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:46:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:46:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:46:20.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:46:20.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:46:20.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:46:20.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:46:20.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:46:20.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:46:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:46:20.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:46:20.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:46:20.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:46:20.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:46:20.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:46:20.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:46:20.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:46:20.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:46:20.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:46:20.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:46:20.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:46:20.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:46:20.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:46:20.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:46:20.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:46:20.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:46:20.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:46:20.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:46:20.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:46:20.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:46:20.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:46:20.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:46:20.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:46:20.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:46:20.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:46:20.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:46:20.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:46:20.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:46:20.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:46:20.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:46:20.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:46:20.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_jerasure_appends td/osd-scrub-repair 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:815: TEST_corrupt_and_repair_jerasure_appends: corrupt_and_repair_jerasure td/osd-scrub-repair false 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:798: corrupt_and_repair_jerasure: local dir=td/osd-scrub-repair 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:799: corrupt_and_repair_jerasure: local allow_overwrites=false 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:800: corrupt_and_repair_jerasure: local poolname=ecpool 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:802: corrupt_and_repair_jerasure: run_mon td/osd-scrub-repair a 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:46:20.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:46:20.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:46:20.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:46:20.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:46:20.493 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:46:20.493 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.493 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:20.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:46:20.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:46:20.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:46:20.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:46:20.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:46:20.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:46:20.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:46:20.523 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:46:20.523 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:46:20.523 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:46:20.523 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:46:20.523 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.524 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.524 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:46:20.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:46:20.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:46:20.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:46:20.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:46:20.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:46:20.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:46:20.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:46:20.580 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:46:20.580 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:46:20.580 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:46:20.581 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:46:20.581 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.581 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.581 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:46:20.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:46:20.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:46:20.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:803: corrupt_and_repair_jerasure: run_mgr td/osd-scrub-repair x 2026-02-17T05:46:20.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:46:20.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:46:20.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:46:20.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:46:20.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:46:20.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:46:20.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:46:20.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:46:20.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:46:20.732 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:46:20.732 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.732 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:20.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:46:20.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:46:20.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: seq 0 3 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 0 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:46:20.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:46:20.757 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:46:20.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:46:20.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:46:20.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:46:20.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:46:20.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7c776fe8-8338-42cb-bc2c-3a0f1f7d1a72 2026-02-17T05:46:20.760 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 7c776fe8-8338-42cb-bc2c-3a0f1f7d1a72 2026-02-17T05:46:20.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 7c776fe8-8338-42cb-bc2c-3a0f1f7d1a72' 2026-02-17T05:46:20.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:46:20.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCsAJRpeWxOLRAABeQRGI0SZhbGyp1moWPjOA== 2026-02-17T05:46:20.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCsAJRpeWxOLRAABeQRGI0SZhbGyp1moWPjOA=="}' 2026-02-17T05:46:20.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7c776fe8-8338-42cb-bc2c-3a0f1f7d1a72 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:46:20.858 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:46:20.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:46:20.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCsAJRpeWxOLRAABeQRGI0SZhbGyp1moWPjOA== --osd-uuid 7c776fe8-8338-42cb-bc2c-3a0f1f7d1a72 2026-02-17T05:46:20.882 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:20.869+0000 7fe5be4bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:20.884 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:20.869+0000 7fe5be4bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:20.885 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:20.873+0000 7fe5be4bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:20.886 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:20.873+0000 7fe5be4bf8c0 -1 bdev(0x560e506afc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:46:20.886 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:20.873+0000 7fe5be4bf8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:46:23.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:46:23.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:46:23.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:46:23.494 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:46:23.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:46:23.682 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:46:23.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:46:23.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:46:23.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:46:23.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:46:23.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:46:23.705 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:23.693+0000 7f082a7518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:23.707 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:23.693+0000 7f082a7518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:23.708 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:23.693+0000 7f082a7518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:23.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:46:23.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:46:23.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:46:23.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:46:23.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:46:23.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:23.835 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:46:23.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:46:23.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:23.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:46:23.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:24.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:24.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:24.991 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:46:24.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:46:24.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:24.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:46:25.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:25.179 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:25.165+0000 7f082a7518c0 -1 Falling back to public interface 2026-02-17T05:46:26.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:26.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:26.143 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:46:26.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:46:26.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:26.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:46:26.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:27.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:27.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:27.296 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:46:27.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:46:27.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:27.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:46:27.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:28.163 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:28.149+0000 7f082a7518c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:46:28.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:28.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:28.449 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:46:28.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:46:28.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:28.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:46:28.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:29.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:29.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:29.602 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:46:29.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:46:29.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:29.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:46:29.756 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/78922569,v1:127.0.0.1:6803/78922569] [v2:127.0.0.1:6804/78922569,v1:127.0.0.1:6805/78922569] exists,up 7c776fe8-8338-42cb-bc2c-3a0f1f7d1a72 2026-02-17T05:46:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:46:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:46:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:46:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:46:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 1 2026-02-17T05:46:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:46:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:46:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:46:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:46:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:46:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:46:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:46:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:46:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:46:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:46:29.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:46:29.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:46:29.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:46:29.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:46:29.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:46:29.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:46:29.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:46:29.767 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:46:29.767 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:29.767 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:29.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:29.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:29.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:46:29.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:46:29.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:46:29.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:46:29.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:46:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:46:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:46:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:46:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:46:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:46:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:46:29.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:46:29.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:46:29.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:46:29.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 82abfaff-2ed2-4dc4-bfd4-4016a153d276' 2026-02-17T05:46:29.771 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:46:29.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:46:29.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC1AJRpBA1rLRAArbWg7TgXaDJ7TuemsU1Ieg== 2026-02-17T05:46:29.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC1AJRpBA1rLRAArbWg7TgXaDJ7TuemsU1Ieg=="}' 2026-02-17T05:46:29.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 82abfaff-2ed2-4dc4-bfd4-4016a153d276 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:46:29.924 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:46:29.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:46:29.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC1AJRpBA1rLRAArbWg7TgXaDJ7TuemsU1Ieg== --osd-uuid 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:46:29.950 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:29.937+0000 7f53ae8908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:29.952 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:29.937+0000 7f53ae8908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:29.953 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:29.941+0000 7f53ae8908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:29.953 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:29.941+0000 7f53ae8908c0 -1 bdev(0x55ef7d90dc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:46:29.953 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:29.941+0000 7f53ae8908c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:46:32.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:46:32.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:46:32.211 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:46:32.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:46:32.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:46:32.396 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:46:32.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:46:32.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:46:32.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:46:32.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:46:32.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:46:32.420 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:32.405+0000 7efce1d4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:32.422 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:32.409+0000 7efce1d4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:32.424 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:32.409+0000 7efce1d4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:46:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:46:32.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:32.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:32.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:33.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:33.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:33.704 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:46:33.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:46:33.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:33.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:33.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:33.927 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:33.913+0000 7efce1d4c8c0 -1 Falling back to public interface 2026-02-17T05:46:34.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:34.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:34.855 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:46:34.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:46:34.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:34.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:35.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:36.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:36.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:36.006 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:46:36.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:46:36.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:36.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:36.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:36.962 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:36.949+0000 7efce1d4c8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:46:37.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:37.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:37.157 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:46:37.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:46:37.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:37.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:37.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:38.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:38.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:38.314 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:46:38.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:46:38.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:38.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:46:38.466 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2510840897,v1:127.0.0.1:6811/2510840897] [v2:127.0.0.1:6812/2510840897,v1:127.0.0.1:6813/2510840897] exists,up 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:46:38.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:46:38.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:46:38.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 2 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:46:38.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:38.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:46:38.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:46:38.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=46d1113f-9799-4749-9449-47748792a70a 2026-02-17T05:46:38.474 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 46d1113f-9799-4749-9449-47748792a70a 2026-02-17T05:46:38.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 46d1113f-9799-4749-9449-47748792a70a' 2026-02-17T05:46:38.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:46:38.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC+AJRpXvpEHBAAyu2rPVYRQgIxhGkLjCyJUQ== 2026-02-17T05:46:38.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC+AJRpXvpEHBAAyu2rPVYRQgIxhGkLjCyJUQ=="}' 2026-02-17T05:46:38.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 46d1113f-9799-4749-9449-47748792a70a -i td/osd-scrub-repair/2/new.json 2026-02-17T05:46:38.641 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:46:38.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:46:38.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+AJRpXvpEHBAAyu2rPVYRQgIxhGkLjCyJUQ== --osd-uuid 46d1113f-9799-4749-9449-47748792a70a 2026-02-17T05:46:38.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:38.653+0000 7f2725fe38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:38.668 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:38.653+0000 7f2725fe38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:38.669 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:38.653+0000 7f2725fe38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:38.669 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:38.657+0000 7f2725fe38c0 -1 bdev(0x560215a9fc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:46:38.669 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:38.657+0000 7f2725fe38c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:46:40.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:46:40.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:46:40.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:46:40.963 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:46:40.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:46:41.155 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:46:41.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:46:41.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:46:41.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:46:41.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:46:41.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:46:41.179 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:41.165+0000 7fb47a7dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:41.180 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:41.165+0000 7fb47a7dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:41.181 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:41.169+0000 7fb47a7dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:41.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:46:41.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:46:41.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:46:41.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:46:41.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:46:41.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:41.311 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:46:41.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:46:41.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:41.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:46:41.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:42.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:42.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:42.464 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:46:42.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:46:42.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:42.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:46:42.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:42.651 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:42.637+0000 7fb47a7dc8c0 -1 Falling back to public interface 2026-02-17T05:46:43.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:43.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:43.618 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:46:43.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:46:43.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:43.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:46:43.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:44.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:44.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:44.768 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:46:44.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:46:44.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:44.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:46:44.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:45.637 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:45.624+0000 7fb47a7dc8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:46:45.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:45.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:45.921 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:46:45.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:46:45.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:45.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:46:46.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:47.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:47.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:47.076 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:46:47.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:46:47.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:47.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:46:47.227 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4272591729,v1:127.0.0.1:6819/4272591729] [v2:127.0.0.1:6820/4272591729,v1:127.0.0.1:6821/4272591729] exists,up 46d1113f-9799-4749-9449-47748792a70a 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 3 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:46:47.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:46:47.230 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:46:47.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:46:47.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:46:47.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:46:47.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:46:47.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:46:47.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:46:47.233 INFO:tasks.workunit.client.0.trial074.stdout:add osd3 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:46:47.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb' 2026-02-17T05:46:47.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:46:47.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDHAJRpwVoHDhAAZCaINLeV7S286m9E9XJObg== 2026-02-17T05:46:47.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDHAJRpwVoHDhAAZCaINLeV7S286m9E9XJObg=="}' 2026-02-17T05:46:47.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb -i td/osd-scrub-repair/3/new.json 2026-02-17T05:46:47.397 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:46:47.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T05:46:47.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDHAJRpwVoHDhAAZCaINLeV7S286m9E9XJObg== --osd-uuid 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:46:47.430 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:47.420+0000 7fb267bad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:47.432 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:47.420+0000 7fb267bad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:47.433 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:47.420+0000 7fb267bad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:47.433 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:47.424+0000 7fb267bad8c0 -1 bdev(0x55866ededc00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T05:46:47.434 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:47.424+0000 7fb267bad8c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T05:46:49.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T05:46:49.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:46:49.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T05:46:49.712 INFO:tasks.workunit.client.0.trial074.stdout:adding osd3 key to auth repository 2026-02-17T05:46:49.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:46:49.904 INFO:tasks.workunit.client.0.trial074.stdout:start osd.3 2026-02-17T05:46:49.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T05:46:49.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:46:49.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:46:49.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:46:49.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:46:49.928 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:49.916+0000 7f43a66f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:49.929 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:49.920+0000 7f43a66f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:49.930 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:49.920+0000 7f43a66f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:46:50.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T05:46:50.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:46:50.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:46:50.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:46:50.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:46:50.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:50.057 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:46:50.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:46:50.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:50.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:46:50.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:51.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:51.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:51.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:46:51.205 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:46:51.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:51.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:46:51.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:51.443 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:51.432+0000 7f43a66f88c0 -1 Falling back to public interface 2026-02-17T05:46:52.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:52.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:52.359 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:46:52.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:46:52.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:52.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:46:52.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:53.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:53.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:53.511 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:46:53.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:46:53.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:53.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:46:53.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:54.420 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:46:54.408+0000 7f43a66f88c0 -1 osd.3 0 log_to_monitors true 2026-02-17T05:46:54.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:54.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:54.659 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:46:54.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:46:54.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:54.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:46:54.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:46:55.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:46:55.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:46:55.815 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:46:55.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:46:55.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:46:55.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:46:55.967 INFO:tasks.workunit.client.0.trial074.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3958908272,v1:127.0.0.1:6827/3958908272] [v2:127.0.0.1:6828/3958908272,v1:127.0.0.1:6829/3958908272] exists,up 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:46:55.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:46:55.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:46:55.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:46:55.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:807: corrupt_and_repair_jerasure: create_rbd_pool 2026-02-17T05:46:55.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:46:56.107 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:46:56.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:46:56.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:46:56.329 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:46:56.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:46:57.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:46:57.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: corrupt_and_repair_jerasure: wait_for_clean 2026-02-17T05:46:57.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:46:57.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:46:57.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:46:57.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:46:57.687 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:46:57.687 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:46:57.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:46:57.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:46:57.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:46:57.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:46:57.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:46:57.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:46:57.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:46:57.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:46:57.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:46:57.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:46:57.948 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:46:57.948 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:46:57.948 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:46:57.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:46:57.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:57.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:46:58.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T05:46:58.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T05:46:58.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T05:46:58.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:58.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:46:58.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672965 2026-02-17T05:46:58.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672965 2026-02-17T05:46:58.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2026-02-17T05:46:58.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:58.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:46:58.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T05:46:58.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T05:46:58.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509444' 2026-02-17T05:46:58.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:46:58.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:46:58.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345922 2026-02-17T05:46:58.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345922 2026-02-17T05:46:58.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509444 3-85899345922' 2026-02-17T05:46:58.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:58.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T05:46:58.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:58.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:46:58.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T05:46:58.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:58.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T05:46:58.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T05:46:58.236 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T05:46:58.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:46:58.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T05:46:58.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:46:59.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:46:59.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:46:59.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T05:46:59.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:59.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672965 2026-02-17T05:46:59.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:59.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:46:59.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672965 2026-02-17T05:46:59.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:59.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672965 2026-02-17T05:46:59.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-17T05:46:59.554 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672965 2026-02-17T05:46:59.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:46:59.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672965 2026-02-17T05:46:59.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:59.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T05:46:59.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:59.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:46:59.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T05:46:59.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:59.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T05:46:59.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T05:46:59.713 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509444 2026-02-17T05:46:59.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:46:59.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T05:46:59.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:46:59.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-85899345922 2026-02-17T05:46:59.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:46:59.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:46:59.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-85899345922 2026-02-17T05:46:59.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:46:59.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345922 2026-02-17T05:46:59.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 85899345922' 2026-02-17T05:46:59.878 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 85899345922 2026-02-17T05:46:59.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:47:00.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345922 -lt 85899345922 2026-02-17T05:47:00.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:47:00.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:00.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:00.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:47:00.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:47:00.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:47:00.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:47:00.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:47:00.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:47:00.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:47:00.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:47:00.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:47:00.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:47:00.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:00.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:810: corrupt_and_repair_jerasure: create_ec_pool ecpool false k=2 m=2 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T05:47:00.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T05:47:00.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T05:47:00.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T05:47:01.031 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T05:47:01.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:47:02.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T05:47:02.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T05:47:02.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:47:02.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:47:02.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:47:02.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:47:02.045 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:47:02.045 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:47:02.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:47:02.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:47:02.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:47:02.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:47:02.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:47:02.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:47:02.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:47:02.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:47:02.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:47:02.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:47:02.301 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:02.301 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:02.301 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:47:02.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:47:02.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:02.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:47:02.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T05:47:02.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T05:47:02.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T05:47:02.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:02.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:47:02.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672967 2026-02-17T05:47:02.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672967 2026-02-17T05:47:02.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967' 2026-02-17T05:47:02.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:02.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:47:02.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509446 2026-02-17T05:47:02.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509446 2026-02-17T05:47:02.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967 2-64424509446' 2026-02-17T05:47:02.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:02.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:47:02.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345924 2026-02-17T05:47:02.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345924 2026-02-17T05:47:02.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967 2-64424509446 3-85899345924' 2026-02-17T05:47:02.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:02.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T05:47:02.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:02.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:47:02.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T05:47:02.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:02.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T05:47:02.590 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836489 2026-02-17T05:47:02.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T05:47:02.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:02.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T05:47:02.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:47:03.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:47:03.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:03.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T05:47:03.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:03.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672967 2026-02-17T05:47:03.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:03.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:47:03.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672967 2026-02-17T05:47:03.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:03.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672967 2026-02-17T05:47:03.906 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672967 2026-02-17T05:47:03.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-17T05:47:03.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:47:04.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-17T05:47:04.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:04.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509446 2026-02-17T05:47:04.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:04.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:47:04.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509446 2026-02-17T05:47:04.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:04.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509446 2026-02-17T05:47:04.062 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509446 2026-02-17T05:47:04.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-17T05:47:04.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:47:04.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-17T05:47:04.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:04.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-85899345924 2026-02-17T05:47:04.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:04.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:47:04.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-85899345924 2026-02-17T05:47:04.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:04.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345924 2026-02-17T05:47:04.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 85899345924' 2026-02-17T05:47:04.222 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 85899345924 2026-02-17T05:47:04.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:47:04.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345924 -lt 85899345924 2026-02-17T05:47:04.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:47:04.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:04.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:04.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:47:04.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:47:04.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:47:04.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:47:04.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:47:04.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:47:04.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:47:04.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:47:04.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:47:04.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:47:04.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:04.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:04.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:47:04.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:47:04.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:47:04.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T05:47:04.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:811: corrupt_and_repair_jerasure: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T05:47:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T05:47:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T05:47:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T05:47:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T05:47:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T05:47:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T05:47:04.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T05:47:04.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T05:47:04.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T05:47:05.110 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T05:47:05.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T05:47:05.311 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T05:47:05.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T05:47:05.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T05:47:05.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:47:05.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T05:47:05.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:47:05.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:47:05.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:47:05.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:47:05.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T05:47:05.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T05:47:05.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T05:47:05.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T05:47:05.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T05:47:05.507 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:47:05.507 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T05:47:05.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T05:47:05.660 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:05.660 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:05.660 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T05:47:05.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 0 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('1' '2' '0') 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=1 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=2 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:47:05.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:47:05.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:47:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:47:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:47:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:47:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:47:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:47:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:47:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:47:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:47:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:47:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:47:06.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:47:07.611 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:47:08.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:47:08.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:47:08.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:47:08.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:47:08.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:47:08.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:47:08.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:47:08.159 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:47:08.159 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:47:08.159 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:47:08.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:08.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:08.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:47:08.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:47:08.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:47:08.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:47:08.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:47:08.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:47:08.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:47:08.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:47:08.182 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:08.172+0000 7f144a17d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:08.183 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:08.172+0000 7f144a17d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:08.184 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:08.172+0000 7f144a17d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:47:08.311 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:47:08.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:08.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:08.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:09.443 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:09.432+0000 7f144a17d8c0 -1 Falling back to public interface 2026-02-17T05:47:09.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:09.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:09.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:47:09.466 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:09.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:09.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:09.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:10.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:10.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:10.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:47:10.618 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:10.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:10.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:10.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:11.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:11.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:11.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:47:11.772 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:47:11.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:11.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:11.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:12.438 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:12.428+0000 7f144a17d8c0 -1 osd.3 36 log_to_monitors true 2026-02-17T05:47:12.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:12.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:12.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:47:12.927 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:47:12.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:12.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:13.080 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [20,36) [v2:127.0.0.1:6826/3084665921,v1:127.0.0.1:6827/3084665921] [v2:127.0.0.1:6828/3084665921,v1:127.0.0.1:6829/3084665921] exists,up 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:47:13.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:47:13.082 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:47:13.083 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:47:13.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:47:13.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:47:13.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:47:13.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:47:13.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:47:13.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:47:13.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:47:13.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:47:13.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:47:13.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:47:13.334 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:13.334 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:13.335 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:47:13.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:47:13.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:13.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:47:13.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T05:47:13.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T05:47:13.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T05:47:13.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:13.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:47:13.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672971 2026-02-17T05:47:13.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672971 2026-02-17T05:47:13.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672971' 2026-02-17T05:47:13.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:13.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:47:13.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509449 2026-02-17T05:47:13.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509449 2026-02-17T05:47:13.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-64424509449' 2026-02-17T05:47:13.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:13.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:47:13.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691842 2026-02-17T05:47:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691842 2026-02-17T05:47:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-64424509449 3-171798691842' 2026-02-17T05:47:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:13.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T05:47:13.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:13.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:47:13.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T05:47:13.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:13.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T05:47:13.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T05:47:13.623 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836492 2026-02-17T05:47:13.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:13.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836492 2026-02-17T05:47:13.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:47:14.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:47:14.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836492 2026-02-17T05:47:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:47:15.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:47:15.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:16.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836492 2026-02-17T05:47:16.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:16.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672971 2026-02-17T05:47:16.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:47:16.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672971 2026-02-17T05:47:16.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:16.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672971 2026-02-17T05:47:16.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-17T05:47:16.092 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672971 2026-02-17T05:47:16.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:47:16.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-17T05:47:16.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:16.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509449 2026-02-17T05:47:16.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:16.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:47:16.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509449 2026-02-17T05:47:16.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:16.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509449 2026-02-17T05:47:16.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-17T05:47:16.255 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509449 2026-02-17T05:47:16.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:47:16.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-17T05:47:16.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:16.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-171798691842 2026-02-17T05:47:16.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:16.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:47:16.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-171798691842 2026-02-17T05:47:16.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:16.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691842 2026-02-17T05:47:16.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 171798691842' 2026-02-17T05:47:16.412 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 171798691842 2026-02-17T05:47:16.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:47:16.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-17T05:47:16.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:47:16.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:16.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:16.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:47:16.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:47:16.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:47:16.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:47:16.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:47:16.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:47:16.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:47:16.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:47:16.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:47:16.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:47:16.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:16.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:17.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:47:17.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:47:17.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:47:17.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:47:17.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:47:17.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:47:17.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:47:17.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:47:17.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:47:17.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:47:17.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:47:17.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:47:17.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:17.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:17.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:17.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:17.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:17.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:47:17.535 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:47:17.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:17.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:17.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:01.020331+0000 '>' 2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:17.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:18.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:18.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:18.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:18.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:18.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:18.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:18.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:18.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:01.020331+0000 '>' 2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:18.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:19.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:19.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:19.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:19.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:19.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:19.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:19.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:19.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:01.020331+0000 '>' 2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:19.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:20.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:20.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:20.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:20.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:20.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:20.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:20.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:21.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:01.020331+0000 '>' 2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:21.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:22.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:22.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:22.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:22.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:22.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:22.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:22.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:22.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:01.020331+0000 '>' 2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:22.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:23.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:23.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:23.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:23.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:23.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:23.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:23.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:18.357864+0000 '>' 2026-02-17T05:47:01.020331+0000 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:47:23.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:47:23.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:47:23.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:47:23.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:47:23.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:47:23.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:47:24.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:47:25.077 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:47:25.078 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:47:25.078 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:47:25.078 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:47:25.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:47:25.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:47:25.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:47:25.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:47:25.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:47:25.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:47:25.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:47:25.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:47:25.625 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:47:25.625 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:47:25.625 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:47:25.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:25.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:25.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:47:25.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:47:25.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:47:25.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:47:25.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:47:25.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:47:25.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:47:25.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:47:25.639 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:25.628+0000 7f89deedc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:25.641 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:25.628+0000 7f89deedc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:25.642 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:25.632+0000 7f89deedc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:25.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:26.924 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:26.912+0000 7f89deedc8c0 -1 Falling back to public interface 2026-02-17T05:47:26.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:26.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:26.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:47:26.932 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:26.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:26.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:27.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:28.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:28.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:28.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:47:28.085 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:28.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:28.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:28.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:29.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:29.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:29.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:47:29.238 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:47:29.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:29.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:29.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:29.889 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:29.875+0000 7f89deedc8c0 -1 osd.3 42 log_to_monitors true 2026-02-17T05:47:30.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:30.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:30.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:47:30.391 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:47:30.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:30.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:30.541 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 46 up_thru 46 down_at 43 last_clean_interval [40,42) [v2:127.0.0.1:6826/4021344415,v1:127.0.0.1:6827/4021344415] [v2:127.0.0.1:6828/4021344415,v1:127.0.0.1:6829/4021344415] exists,up 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:47:30.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:47:30.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:47:30.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:47:30.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:47:30.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:47:30.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:47:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:47:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:47:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:47:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:47:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:47:30.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:47:30.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:47:30.804 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:30.804 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:30.805 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:47:30.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:47:30.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:30.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:47:30.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T05:47:30.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T05:47:30.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T05:47:30.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:30.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:47:30.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672975 2026-02-17T05:47:30.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672975 2026-02-17T05:47:30.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672975' 2026-02-17T05:47:30.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:30.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:47:31.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509453 2026-02-17T05:47:31.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509453 2026-02-17T05:47:31.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509453' 2026-02-17T05:47:31.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:31.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:47:31.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=197568495618 2026-02-17T05:47:31.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 197568495618 2026-02-17T05:47:31.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-64424509453 3-197568495618' 2026-02-17T05:47:31.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:31.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T05:47:31.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:31.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:47:31.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T05:47:31.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:31.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T05:47:31.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T05:47:31.095 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836497 2026-02-17T05:47:31.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:31.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-17T05:47:31.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:31.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672975 2026-02-17T05:47:31.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:31.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:47:31.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672975 2026-02-17T05:47:31.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:31.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672975 2026-02-17T05:47:31.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2026-02-17T05:47:31.259 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672975 2026-02-17T05:47:31.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:47:31.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672975 -lt 42949672975 2026-02-17T05:47:31.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:31.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509453 2026-02-17T05:47:31.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:31.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:47:31.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509453 2026-02-17T05:47:31.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:31.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509453 2026-02-17T05:47:31.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509453' 2026-02-17T05:47:31.419 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509453 2026-02-17T05:47:31.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:47:31.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509452 -lt 64424509453 2026-02-17T05:47:31.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:47:32.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:47:32.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:47:32.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509452 -lt 64424509453 2026-02-17T05:47:32.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:47:33.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:47:33.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:47:33.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509454 -lt 64424509453 2026-02-17T05:47:33.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:33.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-197568495618 2026-02-17T05:47:33.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:33.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:47:33.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-197568495618 2026-02-17T05:47:33.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:33.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=197568495618 2026-02-17T05:47:33.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 197568495618' 2026-02-17T05:47:33.884 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 197568495618 2026-02-17T05:47:33.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:47:34.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 197568495618 -lt 197568495618 2026-02-17T05:47:34.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:47:34.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:34.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:34.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:47:34.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:47:34.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:47:34.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:47:34.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:47:34.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:47:34.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:47:34.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:47:34.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:47:34.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:47:34.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:34.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:34.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:47:34.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:47:34.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:47:34.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:47:34.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:47:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 1 2026-02-17T05:47:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:47:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:47:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=1 2026-02-17T05:47:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:47:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:47:34.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:47:34.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:47:34.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:47:34.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:47:34.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:47:35.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:47:35.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:47:35.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:47:35.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:47:35.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:47:35.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:47:35.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:47:35.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:47:36.523 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:47:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:47:37.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:47:37.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:47:37.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:47:37.071 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:47:37.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:47:37.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:47:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:47:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:47:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:47:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:47:37.094 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:37.083+0000 7f6498cb38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:37.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:37.083+0000 7f6498cb38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:37.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:37.083+0000 7f6498cb38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:37.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:47:37.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:38.336 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:38.323+0000 7f6498cb38c0 -1 Falling back to public interface 2026-02-17T05:47:38.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:38.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:38.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:47:38.374 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:38.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:38.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:47:38.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:39.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:39.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:39.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:47:39.524 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:39.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:39.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:47:39.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:40.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:40.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:40.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:47:40.673 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:47:40.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:40.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:47:40.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:41.330 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:41.319+0000 7f6498cb38c0 -1 osd.1 47 log_to_monitors true 2026-02-17T05:47:41.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:41.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:41.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:47:41.824 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:47:41.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:41.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 48 last_clean_interval [10,47) [v2:127.0.0.1:6810/1095468258,v1:127.0.0.1:6811/1095468258] [v2:127.0.0.1:6812/1095468258,v1:127.0.0.1:6813/1095468258] exists,up 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:47:41.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:47:41.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:47:41.974 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:47:41.974 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:47:41.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:47:41.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:47:41.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:47:42.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:47:42.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:47:42.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:47:42.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:47:42.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:47:42.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:47:42.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:47:42.235 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:42.235 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:42.235 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:47:42.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:47:42.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:42.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:47:42.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836500 2026-02-17T05:47:42.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836500 2026-02-17T05:47:42.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500' 2026-02-17T05:47:42.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:42.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:47:42.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332098 2026-02-17T05:47:42.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332098 2026-02-17T05:47:42.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-219043332098' 2026-02-17T05:47:42.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:42.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:47:42.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509457 2026-02-17T05:47:42.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509457 2026-02-17T05:47:42.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-219043332098 2-64424509457' 2026-02-17T05:47:42.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:42.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:47:42.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=197568495621 2026-02-17T05:47:42.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 197568495621 2026-02-17T05:47:42.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-219043332098 2-64424509457 3-197568495621' 2026-02-17T05:47:42.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:42.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836500 2026-02-17T05:47:42.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:42.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:47:42.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836500 2026-02-17T05:47:42.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:42.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836500 2026-02-17T05:47:42.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-17T05:47:42.525 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836500 2026-02-17T05:47:42.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:42.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836500 2026-02-17T05:47:42.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:47:43.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:47:43.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:43.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-17T05:47:43.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:43.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332098 2026-02-17T05:47:43.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:43.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:47:43.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332098 2026-02-17T05:47:43.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:43.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332098 2026-02-17T05:47:43.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332098' 2026-02-17T05:47:43.841 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 219043332098 2026-02-17T05:47:43.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:47:44.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332098 -lt 219043332098 2026-02-17T05:47:44.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:44.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509457 2026-02-17T05:47:44.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:44.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:47:44.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509457 2026-02-17T05:47:44.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:44.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509457 2026-02-17T05:47:44.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509457' 2026-02-17T05:47:44.011 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509457 2026-02-17T05:47:44.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:47:44.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509457 -lt 64424509457 2026-02-17T05:47:44.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:44.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-197568495621 2026-02-17T05:47:44.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:44.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:47:44.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-197568495621 2026-02-17T05:47:44.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:44.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=197568495621 2026-02-17T05:47:44.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 197568495621' 2026-02-17T05:47:44.162 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 197568495621 2026-02-17T05:47:44.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:47:44.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 197568495621 -lt 197568495621 2026-02-17T05:47:44.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:47:44.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:44.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:44.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:47:44.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:47:44.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:47:44.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:47:44.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:47:44.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:47:44.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:47:44.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:47:44.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:47:44.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:47:44.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:47:44.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:47:44.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:47:44.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:47:44.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:47:44.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:47:44.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:47:44.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:47:44.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:47:44.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:47:44.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:47:44.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:47:44.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:47:44.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:47:44.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:44.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:44.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:44.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:45.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:45.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:47:45.262 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:47:45.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:45.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:47:45.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:45.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:45.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:18.357864+0000 '>' 2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:45.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:46.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:46.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:46.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:46.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:46.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:46.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:46.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:46.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:18.357864+0000 '>' 2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:46.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:47.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:47.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:47.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:47.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:47.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:47.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:47.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:47.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:18.357864+0000 '>' 2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:47.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:48.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:48.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:48.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:48.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:48.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:48.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:48.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:48.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:18.357864+0000 '>' 2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:48.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:49.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:49.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:49.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:49.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:49.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:49.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:49.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:50.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:18.357864+0000 '>' 2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:50.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:47:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:47:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:47:51.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:47:51.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:47:51.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:47:51.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:47:51.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:45.985696+0000 '>' 2026-02-17T05:47:18.357864+0000 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:47:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:47:52.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:47:52.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:47:52.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:47:52.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:47:52.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:47:52.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:47:52.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:47:52.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:47:52.798 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:47:52.799 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:47:52.799 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:47:52.799 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:47:53.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:47:53.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:47:53.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:47:53.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:47:53.341 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:47:53.342 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:47:53.342 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:47:53.344 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:47:53.344 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:47:53.344 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:47:53.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:47:53.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:47:53.346 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:47:53.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:47:53.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:47:53.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:47:53.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:47:53.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:47:53.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:47:53.365 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:53.351+0000 7fb8f63888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:53.367 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:53.355+0000 7fb8f63888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:53.368 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:53.355+0000 7fb8f63888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:47:53.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:47:53.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:47:53.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:47:53.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:47:53.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:47:53.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:53.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:47:53.494 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:47:53.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:53.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:53.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:54.592 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:54.579+0000 7fb8f63888c0 -1 Falling back to public interface 2026-02-17T05:47:54.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:54.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:54.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:47:54.647 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:54.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:54.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:54.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:55.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:55.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:55.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:47:55.797 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:55.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:55.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:55.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:56.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:56.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:47:56.952 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:47:56.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:56.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:57.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:47:57.599 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:47:57.587+0000 7fb8f63888c0 -1 osd.3 52 log_to_monitors true 2026-02-17T05:47:58.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:47:58.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:47:58.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:47:58.103 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:47:58.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:47:58.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:47:58.256 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 56 up_thru 56 down_at 53 last_clean_interval [46,52) [v2:127.0.0.1:6826/4206052481,v1:127.0.0.1:6827/4206052481] [v2:127.0.0.1:6828/4206052481,v1:127.0.0.1:6829/4206052481] exists,up 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:47:58.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:47:58.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:47:58.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:47:58.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:47:58.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:47:58.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:47:58.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:47:58.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:47:58.257 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:47:58.257 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:47:58.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:47:58.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:47:58.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:47:58.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:47:58.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:47:58.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:47:58.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:47:58.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:47:58.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:47:58.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:47:58.516 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:47:58.516 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:47:58.516 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:47:58.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:47:58.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:58.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:47:58.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836504 2026-02-17T05:47:58.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836504 2026-02-17T05:47:58.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504' 2026-02-17T05:47:58.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:58.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:47:58.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332102 2026-02-17T05:47:58.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332102 2026-02-17T05:47:58.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-219043332102' 2026-02-17T05:47:58.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:58.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:47:58.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509461 2026-02-17T05:47:58.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509461 2026-02-17T05:47:58.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-219043332102 2-64424509461' 2026-02-17T05:47:58.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:47:58.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:47:58.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=240518168578 2026-02-17T05:47:58.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 240518168578 2026-02-17T05:47:58.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-219043332102 2-64424509461 3-240518168578' 2026-02-17T05:47:58.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:47:58.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836504 2026-02-17T05:47:58.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:47:58.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:47:58.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836504 2026-02-17T05:47:58.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:47:58.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836504 2026-02-17T05:47:58.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-17T05:47:58.804 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836504 2026-02-17T05:47:58.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:47:58.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836503 -lt 21474836504 2026-02-17T05:47:58.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:47:59.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:47:59.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:00.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836504 -lt 21474836504 2026-02-17T05:48:00.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:00.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332102 2026-02-17T05:48:00.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:00.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:00.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332102 2026-02-17T05:48:00.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332102 2026-02-17T05:48:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332102' 2026-02-17T05:48:00.117 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 219043332102 2026-02-17T05:48:00.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:48:00.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332102 -lt 219043332102 2026-02-17T05:48:00.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:00.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509461 2026-02-17T05:48:00.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:00.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:48:00.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509461 2026-02-17T05:48:00.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:00.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509461 2026-02-17T05:48:00.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509461' 2026-02-17T05:48:00.287 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509461 2026-02-17T05:48:00.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:00.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509461 -lt 64424509461 2026-02-17T05:48:00.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:00.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-240518168578 2026-02-17T05:48:00.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:00.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:48:00.442 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-240518168578 2026-02-17T05:48:00.442 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:00.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=240518168578 2026-02-17T05:48:00.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 240518168578' 2026-02-17T05:48:00.443 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 240518168578 2026-02-17T05:48:00.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:48:00.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 240518168578 -lt 240518168578 2026-02-17T05:48:00.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:48:00.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:00.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:00.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:48:00.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:00.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:00.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:00.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:00.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:00.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:00.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:00.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:48:00.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:00.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:00.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:01.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:48:01.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:48:01.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:48:01.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:48:01.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:48:01.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 1 2 2026-02-17T05:48:01.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=1 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=2 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 133104"' 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 133104' 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:48:01.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 133106"' 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 133106' 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 133104 133106' 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 133104 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/133107: /' 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:48:01.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/133110: /' 2026-02-17T05:48:03.641 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.641 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:03.641 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:03.642 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T05:48:03.643 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:03.644 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:48:03.644 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:03.644 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:03.645 INFO:tasks.workunit.client.0.trial074.stderr:133110: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: start osd.2 2026-02-17T05:48:03.646 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:03.655 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:48:03.656 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:03.657 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:03.657 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:03.657 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:03.658 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:03.658 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:03.658 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:48:03.658 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:03.659 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: start osd.1 2026-02-17T05:48:03.660 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: 2026-02-17T05:48:03.655+0000 7fe7640688c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: 2026-02-17T05:48:03.655+0000 7fe7640688c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: 2026-02-17T05:48:03.655+0000 7fe7640688c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: 0 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:07.948 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: 1 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: 2026-02-17T05:48:04.959+0000 7fe7640688c0 -1 Falling back to public interface 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: 2 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: 3 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:07.949 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133110: 2026-02-17T05:48:07.935+0000 7fe7640688c0 133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: 2026-02-17T05:48:03.663+0000 7f2ef7cfc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: 2026-02-17T05:48:03.663+0000 7f2ef7cfc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: 2026-02-17T05:48:03.667+0000 7f2ef7cfc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:08.084 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: 0 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: 1 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: 2026-02-17T05:48:04.975+0000 7f2ef7cfc8c0 -1 Falling back to public interface 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: 2 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: 3 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:08.085 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133107: 2026-02-17T05:48:08.071+0000 7f2ef7cfc8c0 -1 osd.2 58 log_to_monitors true 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133110: 4 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:08.898 INFO:tasks.workunit.client.0.trial074.stderr:133110: osd.2 up in weight 1 up_from 62 up_thru 62 down_at 59 last_clean_interval [15,58) [v2:127.0.0.1:6810/1826314466,v1:127.0.0.1:6811/1826314466] [v2:127.0.0.1:6812/1826314466,v1:127.0.0.1:6813/1826314466] exists,up 46d1113f-9799-4749-9449-47748792a70a 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:08.899 INFO:tasks.workunit.client.0.trial074.stderr:133110: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:08.900 INFO:tasks.workunit.client.0.trial074.stderr:133110: 1 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: 2 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: 3' 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836507 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836507 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507' 2026-02-17T05:48:08.901 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:08.906 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel-1 osd.1 58 log_to_monitors true 2026-02-17T05:48:08.906 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:08.906 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: 4 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: osd.1 up in weight 1 up_from 63 up_thru 63 down_at 59 last_clean_interval [51,58) [v2:127.0.0.1:6818/1122434063,v1:127.0.0.1:6819/1122434063] [v2:127.0.0.1:6820/1122434063,v1:127.0.0.1:6821/1122434063] exists,up 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:08.907 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:08.908 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: 1 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: 2 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: 3' 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:08.909 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:08.910 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836508 2026-02-17T05:48:08.910 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836508 2026-02-17T05:48:08.910 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508' 2026-02-17T05:48:08.910 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:09.428 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:09.428 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939650 2026-02-17T05:48:09.428 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939650 2026-02-17T05:48:09.428 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-270582939650' 2026-02-17T05:48:09.428 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:09.428 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:09.428 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=266287972354 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 266287972354 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-270582939650 2-266287972354' 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=240518168581 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 240518168581 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-270582939650 2-266287972354 3-240518168581' 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836507 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:09.429 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836507 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836507 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: waiting osd.0 seq 21474836507 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836507 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-270582939650 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-270582939650 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:09.430 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939650 2026-02-17T05:48:09.431 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 270582939650' 2026-02-17T05:48:09.431 INFO:tasks.workunit.client.0.trial074.stderr:133110: waiting osd.1 seq 270582939650 2026-02-17T05:48:09.431 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:48:09.435 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:227pers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:09.435 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939651 2026-02-17T05:48:09.435 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939651 2026-02-17T05:48:09.435 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-270582939651' 2026-02-17T05:48:09.435 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:09.435 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:09.435 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=266287972355 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 266287972355 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-270582939651 2-266287972355' 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=240518168582 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 240518168582 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-270582939651 2-266287972355 3-240518168582' 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836508 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:09.436 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836508 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836508 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: waiting osd.0 seq 21474836508 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-270582939651 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-270582939651 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939651 2026-02-17T05:48:09.437 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 270582939651' 2026-02-17T05:48:09.438 INFO:tasks.workunit.client.0.trial074.stderr:133107: waiting osd.1 seq 270582939651 2026-02-17T05:48:09.438 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:48:12.239 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 270582939651 -lt 270582939650 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-266287972354 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-266287972354 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=266287972354 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 266287972354' 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: waiting osd.2 seq 266287972354 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 266287972354 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 266287972354 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 266287972355 -lt 266287972354 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-240518168581 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-240518168581 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=240518168581 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 240518168581' 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: waiting osd.3 seq 240518168581 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 240518168582 -lt 240518168581 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:48:12.240 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_fo7: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-266287972355 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-266287972355 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=266287972355 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 266287972355' 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: waiting osd.2 seq 266287972355 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 266287972355 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 266287972355 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 266287972355 -lt 266287972355 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-240518168582 2026-02-17T05:48:12.242 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-240518168582 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=240518168582 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 240518168582' 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: waiting osd.3 seq 240518168582 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 240518168582 -lt 240518168582 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:48:12.243 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:133110: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:48:12.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:r_clean: get_num_active_clean 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:133107: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:48:12.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 133106 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:48:12.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:48:12.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:48:12.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:48:12.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:48:12.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:48:12.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:48:12.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:12.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:12.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:12.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:12.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:12.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:48:13.028 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:48:13.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:13.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:48:13.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:13.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:48:13.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:48:13.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:13.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:13.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:13.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:13.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:13.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:13.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:45.985696+0000 '>' 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:13.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:14.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:14.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:14.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:14.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:14.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:14.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:14.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:14.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:45.985696+0000 '>' 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:14.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:15.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:15.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:15.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:15.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:15.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:15.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:15.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:15.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:45.985696+0000 '>' 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:15.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:16.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:16.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:16.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:16.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:16.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:16.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:16.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:16.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:45.985696+0000 '>' 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:16.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:17.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:17.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:17.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:17.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:17.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:17.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:17.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:17.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:45.985696+0000 '>' 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:17.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:18.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:18.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:18.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:18.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:18.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:18.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:18.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:18.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:47:45.985696+0000 '>' 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:18.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:19.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:19.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:19.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:19.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:19.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:19.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:19.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:20.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:48:13.687697+0000 '>' 2026-02-17T05:47:45.985696+0000 2026-02-17T05:48:20.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:48:20.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:48:20.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:20.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:20.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 136156"' 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 136156' 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 136158"' 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 136158' 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 136156 136158' 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:48:20.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:48:20.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 136156 2026-02-17T05:48:20.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/136159: /' 2026-02-17T05:48:20.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:48:20.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/136162: /' 2026-02-17T05:48:22.298 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.298 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:22.298 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:22.299 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING list-attrs 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: _ 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: hinfo_key 2026-02-17T05:48:22.300 INFO:tasks.workunit.client.0.trial074.stderr:136162: snapset 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:22.301 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:22.302 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.302 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:22.302 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:48:22.302 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:22.302 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:22.302 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:22.303 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: _ 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: hinfo_key 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: snapset 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:22.304 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:22.305 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:22.306 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:22.307 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:22.307 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:22.307 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:22.307 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:48:22.307 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T05:48:22.307 INFO:tasks.workunit.client.0.trial074.stderr:136162: start osd.2 2026-02-17T05:48:22.307 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recs.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:22.308 INFO:tasks.workunit.client.0.trial074.stderr:136159: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:22.309 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:48:22.310 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:48:22.310 INFO:tasks.workunit.client.0.trial074.stderr:136159: start osd.1 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: 2026-02-17T05:48:22.302+0000 7f2785cf48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:26.581 INFO:tasks.workunit.client.0.trial074.stderr:136159: 2026-02-17T05:48:22.306+0000 7f2785cf48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: 2026-02-17T05:48:22.306+0000 7f2785cf48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: 0 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:26.582 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: 1 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: 2026-02-17T05:48:23.602+0000 7f2785cf48c0 -1 Falling back to public interface 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: 2 2026-02-17T05:48:26.583 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: 3 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:26.584 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:26.722 INFO:tasks.workunit.client.0.trial074.stderr:136159: 2026-02-17T05:48:26.566+0000 7f2overy_ops 2026-02-17T05:48:26.722 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T05:48:26.722 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: 2026-02-17T05:48:22.302+0000 7f10ec53a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: 2026-02-17T05:48:22.306+0000 7f10ec53a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: 2026-02-17T05:48:22.306+0000 7f10ec53a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:26.723 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: 0 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:26.724 INFO:tasks.workunit.client.0.trial074.stderr:136162: 1 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: 2026-02-17T05:48:23.642+0000 7f10ec53a8c0 -1 Falling back to public interface 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: 2 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:26.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:26.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:26.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:26.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:26.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: 3 2026-02-17T05:48:26.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:26.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:26.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:27.542 INFO:tasks.workunit.client.0.trial074.stderr:136162: 2026-02-17T05:48:26.706+0000 7f1785cf48c0 -1 osd.1 64 log_to_monitors true 2026-02-17T05:48:27.542 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:27.542 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:27.542 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:27.542 INFO:tasks.workunit.client.0.trial074.stderr:136159: 4 2026-02-17T05:48:27.542 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: osd.1 up in weight 1 up_from 68 up_thru 69 down_at 65 last_clean_interval [63,64) [v2:127.0.0.1:6810/494392857,v1:127.0.0.1:6811/494392857] [v2:127.0.0.1:6812/494392857,v1:127.0.0.1:6813/494392857] exists,up 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:27.543 INFO:tasks.workunit.client.0.trial074.stderr:136159: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:27.544 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: 1 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: 2 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: 3' 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836513 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836513 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513' 2026-02-17T05:48:27.545 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ce0ec53a8c0 -1 osd.2 64 log_to_monitors true 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: 4 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: osd.2 up in weight 1 up_from 69 up_thru 62 down_at 65 last_clean_interval [62,64) [v2:127.0.0.1:6818/1488367585,v1:127.0.0.1:6819/1488367585] [v2:127.0.0.1:6820/1488367585,v1:127.0.0.1:6821/1488367585] exists,up 46d1113f-9799-4749-9449-47748792a70a 2026-02-17T05:48:27.557 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:27.558 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: 1 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: 2 2026-02-17T05:48:27.559 INFO:tasks.workunit.client.0.trial074.stderr:136162: 3' 2026-02-17T05:48:27.560 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:27.560 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:27.560 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:27.560 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836514 2026-02-17T05:48:27.560 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836514 2026-02-17T05:48:27.560 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836514' 2026-02-17T05:48:27.560 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:29.082 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:29.082 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776131 2026-02-17T05:48:29.082 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776131 2026-02-17T05:48:29.082 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836514 1-292057776131' 2026-02-17T05:48:29.082 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:29.082 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743427 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743427 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836514 1-292057776131 2-296352743427' 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=240518168588 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 240518168588 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836514 1-292057776131 2-296352743427 3-240518168588' 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:29.083 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836514 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836514 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836514 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: waiting osd.0 seq 21474836514 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836512 -lt 21474836514 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:29.084 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836514 -lt 21474836514 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-292057776131 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-292057776131 2026-02-17T05:48:29.085 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:29.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776131 2026-02-17T05:48:29.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-17T05:48:29.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: waiting osd.1 seq 292057776131 2026-02-17T05:48:29.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:48:29.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-17T05:48:29.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:29.725 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743427 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743427 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743427 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743427' 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: waiting osd.2 seq 296352743427 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743427 -lt 296352743427 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:29.726 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-240518168588 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-240518168588 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=240518168588 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 240518168588' 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: waiting osd.3 seq 240518168588 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 240518168588 -lt 240518168588 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:29.727 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:48:29.728 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:29.728 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:29.728 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:29.728 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:30.219 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(ph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:30.219 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776130 2026-02-17T05:48:30.219 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776130 2026-02-17T05:48:30.219 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-292057776130' 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743426 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743426 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-292057776130 2-296352743426' 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=240518168587 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 240518168587 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-292057776130 2-296352743426 3-240518168587' 2026-02-17T05:48:30.220 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836513 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836513 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836513 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: waiting osd.0 seq 21474836513 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836512 -lt 21474836513 2026-02-17T05:48:30.221 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836512 -lt 21474836513 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836514 -lt 21474836513 2026-02-17T05:48:30.222 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:30.486 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: econtains("stale") | not)' 2026-02-17T05:48:30.486 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' -1 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:30.487 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:48:30.488 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:48:30.889 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_seccho 1-292057776130 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-292057776130 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776130 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 292057776130' 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: waiting osd.1 seq 292057776130 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776131 -lt 292057776130 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743426 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743426 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743426 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743426' 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: waiting osd.2 seq 296352743426 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743427 -lt 296352743426 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-240518168587 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-240518168587 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=240518168587 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 240518168587' 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: waiting osd.3 seq 240518168587 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 240518168588 -lt 240518168587 2026-02-17T05:48:30.890 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:48:30.891 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:30.891 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:30.891 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:48:30.891 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:31.380 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph- 2026-02-17T05:48:31.380 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:48:31.380 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:48:31.380 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:48:31.380 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:48:31.380 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:48:31.380 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:31.381 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:48:31.382 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:48:31.383 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:48:31.383 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' -1 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:48:31.668 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:48:32.396 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:48:32.397 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:32.398 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:48:32.399 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:48:32.399 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:48:32.399 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:48:32.399 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:48:32.399 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:48:32.399 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:48:33.092 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: gehelpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:48:33.092 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:33.093 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:48:33.094 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:48:33.095 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T05:48:33.095 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:48:33.095 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T05:48:33.095 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:48:33.325 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalot_is_making_recovery_progress: ceph --format json status 2026-02-17T05:48:33.325 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:48:33.325 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:48:33.325 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:48:33.325 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T05:48:33.325 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:48:33.325 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:33.326 INFO:tasks.workunit.client.0.trial074.stderr:136159: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:33.327 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:48:33.327 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:48:33.327 INFO:tasks.workunit.client.0.trial074.stderr:136159: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:48:33.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:48:33.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:48:33.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 136158 2026-02-17T05:48:33.426 INFO:tasks.workunit.client.0.trial074.stderr:ne/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:33.426 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:33.426 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:33.426 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:33.426 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:33.426 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:136162: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:48:33.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:48:33.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:48:33.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:48:33.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:48:33.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:48:33.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:48:33.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 1 2026-02-17T05:48:33.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:48:33.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:48:33.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T05:48:33.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=1 2026-02-17T05:48:33.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:48:33.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 139059"' 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 139059' 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 139061"' 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 139061' 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:48:33.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 139059 139061' 2026-02-17T05:48:33.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:48:33.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:48:33.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 139059 2026-02-17T05:48:33.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/139062: /' 2026-02-17T05:48:33.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:33.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/139064: /' 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:35.945 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:35.946 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:35.948 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:35.948 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:35.948 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:48:35.948 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:48:35.948 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:35.948 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:35.948 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: start osd.1 2026-02-17T05:48:35.949 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:48:36.200 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:36.200 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:36.200 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:48:36.200 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:36.200 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:48:36.200 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:48:36.201 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:48:36.202 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:36.203 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:36.203 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:36.203 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:36.203 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:48:36.203 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:36.204 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: start osd.3 2026-02-17T05:48:36.205 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:48:40.500 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:48:40.500 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:48:40.500 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:40.500 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:40.500 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: 2026-02-17T05:48:36.202+0000 7f76775af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: 2026-02-17T05:48:36.202+0000 7f76775af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: 2026-02-17T05:48:36.206+0000 7f76775af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: 0 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: 1 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: 2026-02-17T05:48:37.502+0000 7f76775af8c0 -1 Falling back to public interface 2026-02-17T05:48:40.501 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: 2 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: 3 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:40.502 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:40.536 INFO:tasks.workunit.client.0.trial074.stderr:139062: 2026-02-17T05:48:40.486+0000 7f76775af8c0 139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: 2026-02-17T05:48:35.958+0000 7f23318d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: 2026-02-17T05:48:35.958+0000 7f23318d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: 2026-02-17T05:48:35.958+0000 7f23318d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:40.537 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: 0 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: 1 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: 2026-02-17T05:48:37.514+0000 7f23318d88c0 -1 Falling back to public interface 2026-02-17T05:48:40.538 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: 2 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: 3 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:40.539 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:41.198 INFO:tasks.workunit.client.0.trial074.stderr:139064: 2026-02-17T05:48:40.522+0000 7f23318d88c0 -1 osd.1 70 log_to_monitors true 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: 4 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: osd.1 up in weight 1 up_from 74 up_thru 74 down_at 71 last_clean_interval [68,70) [v2:127.0.0.1:6826/534918730,v1:127.0.0.1:6827/534918730] [v2:127.0.0.1:6828/534918730,v1:127.0.0.1:6829/534918730] exists,up 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:41.199 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:41.200 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: 1 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: 2 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: 3' 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836518 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836518 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518' 2026-02-17T05:48:41.201 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:41.450 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers-1 osd.3 70 log_to_monitors true 2026-02-17T05:48:41.450 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:41.450 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:41.450 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:41.450 INFO:tasks.workunit.client.0.trial074.stderr:139062: 4 2026-02-17T05:48:41.450 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:41.450 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: osd.3 up in weight 1 up_from 74 up_thru 74 down_at 71 last_clean_interval [56,70) [v2:127.0.0.1:6810/1457089902,v1:127.0.0.1:6811/1457089902] [v2:127.0.0.1:6812/1457089902,v1:127.0.0.1:6813/1457089902] exists,up 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: 1 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: 2 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: 3' 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836519 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836519 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519' 2026-02-17T05:48:41.451 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:43.867 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:43.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579906 2026-02-17T05:48:43.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579906 2026-02-17T05:48:43.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-317827579906' 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743430 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743430 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-317827579906 2-296352743430' 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579906 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579906 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-317827579906 2-296352743430 3-317827579906' 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:43.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836518 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836518 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836518 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: waiting osd.0 seq 21474836518 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836518 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:43.869 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836518 2026-02-17T05:48:43.870 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:43.870 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:48:43.870 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:43.870 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836519 -lt 21474836518 2026-02-17T05:48:43.870 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-3178pers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579907 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579907 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-317827579907' 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743431 2026-02-17T05:48:44.129 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743431 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-317827579907 2-296352743431' 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579907 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579907 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-317827579907 2-296352743431 3-317827579907' 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836519 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836519 2026-02-17T05:48:44.130 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836519 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836519' 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: waiting osd.0 seq 21474836519 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836519 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836519 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:48:44.131 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:44.132 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836519 -lt 21474836519 2026-02-17T05:48:44.132 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:44.525 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-27579906 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-317827579906 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579906 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 317827579906' 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: waiting osd.1 seq 317827579906 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579907 -lt 317827579906 2026-02-17T05:48:44.526 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743430 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743430 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743430 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743430' 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: waiting osd.2 seq 296352743430 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743432 -lt 296352743430 2026-02-17T05:48:44.527 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-317827579906 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-317827579906 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579906 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 317827579906' 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: waiting osd.3 seq 317827579906 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:48:44.528 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579907 -lt 317827579906 2026-02-17T05:48:44.529 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:48:44.529 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:44.529 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:44.529 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:48:44.529 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:44.767 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh317827579907 2026-02-17T05:48:44.767 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-317827579907 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579907 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 317827579907' 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: waiting osd.1 seq 317827579907 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743431 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:44.768 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743431 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743431 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743431' 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: waiting osd.2 seq 296352743431 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743432 -lt 296352743431 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-317827579907 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:48:44.769 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-317827579907 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579907 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 317827579907' 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: waiting osd.3 seq 317827579907 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:48:44.770 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:48:44.866 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:44.867 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:44.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:44.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:48:44.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:48:44.868 INFO:tasks.workunit.client.0.trial074.stderr:139064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:48:44.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:48:45.102 INFO:tasks.workunit.client.0.trial074.stderr:s.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:48:45.102 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:48:45.102 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:48:45.102 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:139062: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:48:45.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 139061 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:48:45.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:48:45.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:48:45.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:48:45.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:48:45.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:48:45.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:45.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:45.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:45.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:45.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:45.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:48:45.551 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:45.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:45.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:45.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:48:13.687697+0000 '>' 2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:45.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:46.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:46.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:46.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:46.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:46.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:46.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:46.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:46.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:48:13.687697+0000 '>' 2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:46.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:47.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:47.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:47.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:47.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:47.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:47.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:47.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:48.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:48:13.687697+0000 '>' 2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:48.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:49.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:49.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:49.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:49.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:49.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:49.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:49.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:49.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:48:13.687697+0000 '>' 2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:49.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:50.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:50.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:50.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:50.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:50.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:50.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:50.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:48:13.687697+0000 '>' 2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:48:51.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:48:51.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:48:51.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:48:51.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:48:51.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:48:51.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:48:51.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:48:46.534237+0000 '>' 2026-02-17T05:48:13.687697+0000 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 142086"' 2026-02-17T05:48:51.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 142086' 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 142088"' 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 142088' 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 142086 142088' 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 142086 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/142089: /' 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:51.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/142091: /' 2026-02-17T05:48:53.644 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:53.645 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:48:53.646 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: _ 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: hinfo_key 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: snapset 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:53.647 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:48:53.648 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:53.648 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:48:53.648 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:53.648 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:53.648 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:53.649 INFO:tasks.workunit.client.0.trial074.stderr:142089: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:53.650 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:53.651 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:53.651 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:53.651 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:53.651 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:48:53.651 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:48:53.651 INFO:tasks.workunit.client.0.trial074.stderr:142089: start osd.3 2026-02-17T05:48:53.662 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.662 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:48:53.662 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:48:53.663 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: _ 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: hinfo_key 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: snapset 2026-02-17T05:48:53.664 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:48:53.665 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:48:53.665 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:48:53.665 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:48:53.665 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:48:53.665 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:48:53.665 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:48:53.665 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:48:53.666 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:48:53.667 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:48:53.668 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:48:53.668 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:48:53.668 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:48:53.668 INFO:tasks.workunit.client.0.trial074.stderr:142091: start osd.1 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reclen=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142089: 2026-02-17T05:48:53.653+0000 7f99c56a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:57.412 INFO:tasks.workunit.client.0.trial074.stderr:142089: 2026-02-17T05:48:53.653+0000 7f99c56a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: 2026-02-17T05:48:53.653+0000 7f99c56a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: 0 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:57.413 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: 1 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: 2026-02-17T05:48:55.149+0000 7f99c56a08c0 -1 Falling back to public interface 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:57.414 INFO:tasks.workunit.client.0.trial074.stderr:142089: 2 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: 3 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.415 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:57.976 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T05:48:57.976 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:48:57.976 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:48:57.976 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:48:57.976 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:48:57.976 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:48:57.976 INFO:tasks.workunit.client.0.trial074.stderr:142091: 2026-02-17T05:48:53.669+0000 7fd1800f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: 2026-02-17T05:48:53.669+0000 7fd1800f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: 2026-02-17T05:48:53.669+0000 7fd1800f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:48:57.977 INFO:tasks.workunit.client.0.trial074.stderr:142091: 0 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: 2026-02-17T05:48:54.933+0000 7fd1800f28c0 -1 Falling back to public interface 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: 1 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:57.978 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: 2 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:48:57.979 INFO:tasks.workunit.client.0.trial074.stderr:142091: 3 2026-02-17T05:48:57.980 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:57.980 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:57.980 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:58.893 INFO:tasks.workunit.client.0.trial074.stderr:142091: 2026-02-17T05:48:57.961+0000 7fd.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:48:58.893 INFO:tasks.workunit.client.0.trial074.stderr:142089: 2026-02-17T05:48:58.197+0000 7f99c56a08c0 -1 osd.3 76 log_to_monitors true 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: 4 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: osd.3 up in weight 1 up_from 82 up_thru 82 down_at 77 last_clean_interval [74,76) [v2:127.0.0.1:6826/1067992698,v1:127.0.0.1:6827/1067992698] [v2:127.0.0.1:6828/1067992698,v1:127.0.0.1:6829/1067992698] exists,up 4dcbdd9f-9fb7-4eeb-9cf3-1ca725d596bb 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:58.894 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:58.895 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: 1 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: 2 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: 3' 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836523 2026-02-17T05:48:58.896 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836523 2026-02-17T05:48:58.897 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523' 2026-02-17T05:48:58.897 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers1800f28c0 -1 osd.1 76 log_to_monitors true 2026-02-17T05:48:58.897 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:48:58.897 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:48:58.897 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: 4 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: osd.1 up in weight 1 up_from 80 up_thru 80 down_at 77 last_clean_interval [74,76) [v2:127.0.0.1:6810/3025640289,v1:127.0.0.1:6811/3025640289] [v2:127.0.0.1:6812/3025640289,v1:127.0.0.1:6813/3025640289] exists,up 82abfaff-2ed2-4dc4-bfd4-4016a153d276 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:48:58.898 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:48:58.899 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: 1 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: 2 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: 3' 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836524 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836524 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524' 2026-02-17T05:48:58.900 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:59.270 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalon.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:59.270 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=343597383683 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 343597383683 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-343597383683' 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743436 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743436 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-343597383683 2-296352743436' 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:59.271 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318274 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318274 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-343597383683 2-296352743436 3-352187318274' 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836523 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836523 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836523 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836523' 2026-02-17T05:48:59.272 INFO:tasks.workunit.client.0.trial074.stderr:142089: waiting osd.0 seq 21474836523 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836523 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-343597383683 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-343597383683 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=343597383683 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 343597383683' 2026-02-17T05:48:59.273 INFO:tasks.workunit.client.0.trial074.stderr:142089: waiting osd.1 seq 343597383683 2026-02-17T05:48:59.430 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_e/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:48:59.430 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=343597383682 2026-02-17T05:48:59.430 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 343597383682 2026-02-17T05:48:59.430 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-343597383682' 2026-02-17T05:48:59.430 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:59.430 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743437 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743437 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-343597383682 2-296352743437' 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318275 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318275 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-343597383682 2-296352743437 3-352187318275' 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836524 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:59.431 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836524 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836524 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836524' 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: waiting osd.0 seq 21474836524 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836524 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-343597383682 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-343597383682 2026-02-17T05:48:59.432 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:48:59.433 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=343597383682 2026-02-17T05:48:59.433 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 343597383682' 2026-02-17T05:48:59.433 INFO:tasks.workunit.client.0.trial074.stderr:142091: waiting osd.1 seq 343597383682 2026-02-17T05:48:59.433 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:49:02.227 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpstats: ceph osd last-stat-seq 1 2026-02-17T05:49:02.227 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 343597383683 -lt 343597383683 2026-02-17T05:49:02.227 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743436 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743436 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743436 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743436' 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: waiting osd.2 seq 296352743436 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743435 -lt 296352743436 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743435 -lt 296352743436 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743437 -lt 296352743436 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-352187318274 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-352187318274 2026-02-17T05:49:02.228 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318274 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 352187318274' 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: waiting osd.3 seq 352187318274 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318275 -lt 352187318274 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:02.229 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ers.sh:2277: flush_pg_stats: test 343597383683 -lt 343597383682 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743437 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743437 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743437 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743437' 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: waiting osd.2 seq 296352743437 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743435 -lt 296352743437 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:49:02.238 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743435 -lt 296352743437 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743437 -lt 296352743437 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-352187318275 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-352187318275 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318275 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 352187318275' 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: waiting osd.3 seq 352187318275 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318275 -lt 352187318275 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:49:02.239 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:49:02.559 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standaloceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:49:02.559 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:142089: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:49:02.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 142088 2026-02-17T05:49:02.573 INFO:tasks.workunit.client.0.trial074.stderr:ne/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:49:02.573 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:49:02.573 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:49:02.573 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:49:02.573 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:142091: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:49:02.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:49:02.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:49:02.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:49:02.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:49:02.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:49:02.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:49:02.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:49:02.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:49:02.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:49:02.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:49:02.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:49:02.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:49:02.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:49:02.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:49:02.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:49:02.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:49:02.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:49:02.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:49:02.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:49:02.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:49:02.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:49:02.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:49:02.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:49:02.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:49:02.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:49:02.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:49:02.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:49:02.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:49:02.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:02.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:02.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:49:02.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:49:02.743 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:49:02.743 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:49:02.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:49:02.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:49:02.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:49:02.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:49:02.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:49:02.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:49:02.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:49:02.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:49:02.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:49:02.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:49:02.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:49:02.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:49:02.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:49:02.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:49:02.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:49:02.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:49:02.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:49:02.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:49:02.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:02.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:02.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:49:02.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:49:02.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:49:02.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:49:02.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:49:02.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:02.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:02.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:49:02.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:49:02.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_jerasure_overwrites td/osd-scrub-repair 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_corrupt_and_repair_jerasure_overwrites: '[' true = true ']' 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:820: TEST_corrupt_and_repair_jerasure_overwrites: corrupt_and_repair_jerasure td/osd-scrub-repair true 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:798: corrupt_and_repair_jerasure: local dir=td/osd-scrub-repair 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:799: corrupt_and_repair_jerasure: local allow_overwrites=true 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:800: corrupt_and_repair_jerasure: local poolname=ecpool 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:802: corrupt_and_repair_jerasure: run_mon td/osd-scrub-repair a 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:49:02.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:49:02.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:49:02.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:49:02.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:49:02.782 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:49:02.782 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:02.782 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:02.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:02.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:49:02.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:49:02.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:49:02.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:49:02.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:49:02.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:49:02.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:49:02.813 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:49:02.813 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:49:02.813 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:49:02.814 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:49:02.814 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:02.814 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:02.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:49:02.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:49:02.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:49:02.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:49:02.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:49:02.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:49:02.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:49:02.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:49:02.870 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:49:02.870 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:49:02.870 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:49:02.870 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:49:02.870 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:02.870 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:02.870 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:49:02.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:49:02.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:49:02.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:803: corrupt_and_repair_jerasure: run_mgr td/osd-scrub-repair x 2026-02-17T05:49:02.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:49:02.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:49:02.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:49:02.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:49:02.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:49:02.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:49:03.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:49:03.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:49:03.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:49:03.021 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:49:03.021 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:03.021 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:03.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:03.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:49:03.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:49:03.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: seq 0 3 2026-02-17T05:49:03.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 0 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:49:03.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:03.043 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:03.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:03.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:03.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:49:03.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:49:03.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:49:03.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1f73c6a3-dce7-4026-b1d3-3b0ba4a56ea3 2026-02-17T05:49:03.047 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 1f73c6a3-dce7-4026-b1d3-3b0ba4a56ea3 2026-02-17T05:49:03.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 1f73c6a3-dce7-4026-b1d3-3b0ba4a56ea3' 2026-02-17T05:49:03.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:49:03.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBPAZRp5CidAhAARc51+sv5+7wRJB+uTnAZRw== 2026-02-17T05:49:03.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBPAZRp5CidAhAARc51+sv5+7wRJB+uTnAZRw=="}' 2026-02-17T05:49:03.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1f73c6a3-dce7-4026-b1d3-3b0ba4a56ea3 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:49:03.144 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:49:03.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:49:03.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBPAZRp5CidAhAARc51+sv5+7wRJB+uTnAZRw== --osd-uuid 1f73c6a3-dce7-4026-b1d3-3b0ba4a56ea3 2026-02-17T05:49:03.169 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:03.153+0000 7f7fa76ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:03.170 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:03.153+0000 7f7fa76ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:03.172 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:03.157+0000 7f7fa76ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:03.172 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:03.157+0000 7f7fa76ae8c0 -1 bdev(0x55bd517f5c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:49:03.172 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:03.157+0000 7f7fa76ae8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:49:05.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:49:05.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:49:05.778 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:49:05.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:49:05.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:49:05.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:49:05.964 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:49:05.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:49:05.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:49:05.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:49:05.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:49:05.980 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:05.965+0000 7f2a84e418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:05.981 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:05.965+0000 7f2a84e418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:05.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:05.965+0000 7f2a84e418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:06.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:49:06.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:49:06.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:49:06.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:49:06.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:49:06.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:06.117 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:49:06.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:49:06.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:06.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:49:06.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:07.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:07.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:07.272 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:49:07.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:49:07.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:07.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:49:07.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:07.453 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:07.437+0000 7f2a84e418c0 -1 Falling back to public interface 2026-02-17T05:49:08.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:08.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:08.426 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:49:08.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:49:08.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:08.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:49:08.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:09.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:09.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:09.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:49:09.582 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:49:09.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:09.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:49:09.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:10.452 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:10.437+0000 7f2a84e418c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:49:10.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:10.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:10.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:49:10.738 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:49:10.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:10.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:49:10.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:11.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:11.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:11.892 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:49:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:49:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:49:12.046 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1328792071,v1:127.0.0.1:6803/1328792071] [v2:127.0.0.1:6804/1328792071,v1:127.0.0.1:6805/1328792071] exists,up 1f73c6a3-dce7-4026-b1d3-3b0ba4a56ea3 2026-02-17T05:49:12.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:49:12.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:49:12.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:49:12.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:49:12.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 1 2026-02-17T05:49:12.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:49:12.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:49:12.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:49:12.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:49:12.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:49:12.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:49:12.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:49:12.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:49:12.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:49:12.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:49:12.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:49:12.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:49:12.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:49:12.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:49:12.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:49:12.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:49:12.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:49:12.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:49:12.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:12.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:12.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:12.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:12.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:49:12.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:49:12.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:49:12.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:49:12.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:49:12.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:49:12.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:49:12.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:49:12.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:49:12.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:49:12.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:49:12.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:49:12.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:49:12.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:49:12.060 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:49:12.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 0a05e72b-9b3e-4b52-b609-778f78e1e2e6' 2026-02-17T05:49:12.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:49:12.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBYAZRp/FAMAxAA9+ur5cXMSKwltLuPlg1rfQ== 2026-02-17T05:49:12.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBYAZRp/FAMAxAA9+ur5cXMSKwltLuPlg1rfQ=="}' 2026-02-17T05:49:12.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:49:12.217 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:49:12.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:49:12.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBYAZRp/FAMAxAA9+ur5cXMSKwltLuPlg1rfQ== --osd-uuid 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:49:12.244 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:12.229+0000 7f42d4eaa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:12.245 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:12.229+0000 7f42d4eaa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:12.246 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:12.229+0000 7f42d4eaa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:12.247 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:12.229+0000 7f42d4eaa8c0 -1 bdev(0x55a7ec373c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:49:12.247 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:12.229+0000 7f42d4eaa8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:49:14.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:49:14.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:49:14.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:49:14.530 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:49:14.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:49:14.714 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:49:14.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:49:14.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:49:14.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:49:14.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:49:14.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:49:14.738 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:14.721+0000 7f52b0bd58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:14.740 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:14.725+0000 7f52b0bd58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:14.741 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:14.725+0000 7f52b0bd58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:14.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:49:14.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:49:14.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:49:14.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:49:14.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:49:14.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:14.862 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:49:14.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:49:14.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:14.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:49:15.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:16.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:16.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:16.015 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:49:16.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:49:16.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:16.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:49:16.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:16.233 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:16.217+0000 7f52b0bd58c0 -1 Falling back to public interface 2026-02-17T05:49:17.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:17.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:17.168 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:49:17.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:49:17.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:17.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:49:17.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:18.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:18.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:18.325 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:49:18.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:49:18.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:18.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:49:18.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:19.259 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:19.241+0000 7f52b0bd58c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:49:19.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:19.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:19.484 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:49:19.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:49:19.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:19.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:49:19.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:20.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:20.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:20.637 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:49:20.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:49:20.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:20.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:49:20.788 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/196168821,v1:127.0.0.1:6811/196168821] [v2:127.0.0.1:6812/196168821,v1:127.0.0.1:6813/196168821] exists,up 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 2 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:49:20.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:49:20.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:49:20.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:49:20.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:49:20.790 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:49:20.790 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:20.790 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:20.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:49:20.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:49:20.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:49:20.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=61db70a1-40d3-428d-851a-55fbe9fddac6 2026-02-17T05:49:20.793 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 61db70a1-40d3-428d-851a-55fbe9fddac6 2026-02-17T05:49:20.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 61db70a1-40d3-428d-851a-55fbe9fddac6' 2026-02-17T05:49:20.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:49:20.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBgAZRpIvRVLxAAJre+PMlMeDN7ls+Y9is0xA== 2026-02-17T05:49:20.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBgAZRpIvRVLxAAJre+PMlMeDN7ls+Y9is0xA=="}' 2026-02-17T05:49:20.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 61db70a1-40d3-428d-851a-55fbe9fddac6 -i td/osd-scrub-repair/2/new.json 2026-02-17T05:49:20.951 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:49:20.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:49:20.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBgAZRpIvRVLxAAJre+PMlMeDN7ls+Y9is0xA== --osd-uuid 61db70a1-40d3-428d-851a-55fbe9fddac6 2026-02-17T05:49:20.979 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:20.961+0000 7f328236e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:20.981 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:20.965+0000 7f328236e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:20.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:20.965+0000 7f328236e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:20.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:20.965+0000 7f328236e8c0 -1 bdev(0x55e3d1ca5c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:49:20.982 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:20.965+0000 7f328236e8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:49:23.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:49:23.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:49:23.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:49:23.242 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:49:23.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:49:23.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:49:23.438 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:49:23.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:49:23.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:49:23.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:49:23.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:49:23.462 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:23.445+0000 7f0ee4bd88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:23.464 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:23.449+0000 7f0ee4bd88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:23.465 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:23.449+0000 7f0ee4bd88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:23.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:49:23.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:49:23.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:49:23.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:49:23.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:49:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:49:23.588 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:49:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:23.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:49:23.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:24.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:24.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:24.741 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:49:24.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:49:24.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:24.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:49:24.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:24.985 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:24.969+0000 7f0ee4bd88c0 -1 Falling back to public interface 2026-02-17T05:49:25.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:25.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:25.896 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:49:25.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:49:25.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:25.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:49:26.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:27.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:27.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:27.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:49:27.049 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:49:27.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:27.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:49:27.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:27.986 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:27.969+0000 7f0ee4bd88c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:49:28.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:28.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:28.201 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:49:28.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:49:28.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:28.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:49:28.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:29.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:29.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:29.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:49:29.351 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:49:29.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:29.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:49:29.499 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3252098026,v1:127.0.0.1:6819/3252098026] [v2:127.0.0.1:6820/3252098026,v1:127.0.0.1:6821/3252098026] exists,up 61db70a1-40d3-428d-851a-55fbe9fddac6 2026-02-17T05:49:29.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 3 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:49:29.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:29.504 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:49:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:49:29.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:49:29.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:49:29.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:49:29.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:49:29.506 INFO:tasks.workunit.client.0.trial074.stdout:add osd3 da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:49:29.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 da21098d-9141-419d-b0df-c787e2d8eb35' 2026-02-17T05:49:29.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:49:29.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBpAZRpZR4eHhAAFLwft7qVvBrsJYU9iTU11g== 2026-02-17T05:49:29.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBpAZRpZR4eHhAAFLwft7qVvBrsJYU9iTU11g=="}' 2026-02-17T05:49:29.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new da21098d-9141-419d-b0df-c787e2d8eb35 -i td/osd-scrub-repair/3/new.json 2026-02-17T05:49:29.664 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:49:29.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T05:49:29.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBpAZRpZR4eHhAAFLwft7qVvBrsJYU9iTU11g== --osd-uuid da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:49:29.689 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:29.676+0000 7f3c528c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:29.690 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:29.676+0000 7f3c528c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:29.692 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:29.676+0000 7f3c528c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:29.692 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:29.680+0000 7f3c528c68c0 -1 bdev(0x563ec9873c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T05:49:29.692 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:29.680+0000 7f3c528c68c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T05:49:31.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T05:49:31.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:49:31.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T05:49:31.970 INFO:tasks.workunit.client.0.trial074.stdout:adding osd3 key to auth repository 2026-02-17T05:49:31.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:49:32.163 INFO:tasks.workunit.client.0.trial074.stdout:start osd.3 2026-02-17T05:49:32.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T05:49:32.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:49:32.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:49:32.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:49:32.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:49:32.187 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:32.172+0000 7f3d316fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:32.189 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:32.176+0000 7f3d316fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:32.190 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:32.176+0000 7f3d316fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T05:49:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:49:32.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:49:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:49:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:49:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:32.317 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:49:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:49:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:32.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:32.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:33.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:33.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:33.474 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:49:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:49:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:33.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:33.689 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:33.676+0000 7f3d316fc8c0 -1 Falling back to public interface 2026-02-17T05:49:34.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:34.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:34.628 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:49:34.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:49:34.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:34.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:34.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:35.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:35.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:35.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:49:35.781 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:49:35.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:35.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:35.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:36.696 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:36.684+0000 7f3d316fc8c0 -1 osd.3 0 log_to_monitors true 2026-02-17T05:49:36.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:36.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:36.933 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:49:36.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:49:36.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:36.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:37.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:38.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:38.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:38.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:49:38.083 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:49:38.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:38.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:38.234 INFO:tasks.workunit.client.0.trial074.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3682744708,v1:127.0.0.1:6827/3682744708] [v2:127.0.0.1:6828/3682744708,v1:127.0.0.1:6829/3682744708] exists,up da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:49:38.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:49:38.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:49:38.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:49:38.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:807: corrupt_and_repair_jerasure: create_rbd_pool 2026-02-17T05:49:38.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:49:38.376 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:49:38.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:49:38.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:49:38.595 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:49:38.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:49:39.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:49:39.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: corrupt_and_repair_jerasure: wait_for_clean 2026-02-17T05:49:39.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:49:39.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:49:39.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:49:39.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:49:39.890 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:49:39.890 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:49:39.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:49:39.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:49:39.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:49:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:49:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:49:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:49:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:49:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:49:40.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:49:40.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:49:40.149 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:49:40.149 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:49:40.149 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:49:40.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:49:40.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:40.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:49:40.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T05:49:40.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T05:49:40.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T05:49:40.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:40.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:49:40.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T05:49:40.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T05:49:40.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-17T05:49:40.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:40.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:49:40.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-17T05:49:40.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-17T05:49:40.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574852' 2026-02-17T05:49:40.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:40.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:49:40.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411330 2026-02-17T05:49:40.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411330 2026-02-17T05:49:40.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574852 3-77309411330' 2026-02-17T05:49:40.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:40.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T05:49:40.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:40.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:49:40.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T05:49:40.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:40.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T05:49:40.437 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T05:49:40.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T05:49:40.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:49:40.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T05:49:40.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:49:41.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:49:41.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:49:41.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T05:49:41.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:41.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T05:49:41.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:41.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:49:41.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T05:49:41.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:41.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T05:49:41.754 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705669 2026-02-17T05:49:41.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T05:49:41.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:49:41.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705669 2026-02-17T05:49:41.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:41.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-17T05:49:41.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:41.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:49:41.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-17T05:49:41.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:41.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-17T05:49:41.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-17T05:49:41.919 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574852 2026-02-17T05:49:41.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:42.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-17T05:49:42.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:42.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411330 2026-02-17T05:49:42.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:42.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:49:42.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411330 2026-02-17T05:49:42.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:42.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411330 2026-02-17T05:49:42.082 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 77309411330 2026-02-17T05:49:42.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411330' 2026-02-17T05:49:42.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:49:42.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411330 -lt 77309411330 2026-02-17T05:49:42.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:49:42.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:42.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:42.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:49:42.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:49:42.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:49:42.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:49:42.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:49:42.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:49:42.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:49:42.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:49:42.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:49:42.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:49:42.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:42.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:42.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:49:42.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:49:42.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:49:42.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:810: corrupt_and_repair_jerasure: create_ec_pool ecpool true k=2 m=2 2026-02-17T05:49:42.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T05:49:42.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T05:49:42.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T05:49:42.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T05:49:42.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T05:49:42.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T05:49:42.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T05:49:43.219 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T05:49:43.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:49:44.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T05:49:44.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T05:49:44.442 INFO:tasks.workunit.client.0.trial074.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T05:49:44.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T05:49:44.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:49:44.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:49:44.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:49:44.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:49:44.452 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:49:44.452 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:49:44.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:49:44.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:49:44.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:49:44.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:49:44.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:49:44.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:49:44.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:49:44.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:49:44.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:49:44.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:49:44.724 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:49:44.724 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:49:44.725 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:49:44.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:49:44.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:44.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:49:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T05:49:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T05:49:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T05:49:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:44.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:49:44.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-17T05:49:44.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-17T05:49:44.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671' 2026-02-17T05:49:44.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:44.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:49:44.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574854 2026-02-17T05:49:44.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574854 2026-02-17T05:49:44.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-55834574854' 2026-02-17T05:49:44.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:44.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:49:45.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411332 2026-02-17T05:49:45.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411332 2026-02-17T05:49:45.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-55834574854 3-77309411332' 2026-02-17T05:49:45.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:45.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T05:49:45.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:45.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:49:45.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T05:49:45.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:45.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T05:49:45.021 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836489 2026-02-17T05:49:45.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T05:49:45.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:49:45.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T05:49:45.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:49:46.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:49:46.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:49:46.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T05:49:46.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:46.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-17T05:49:46.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:46.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:49:46.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-17T05:49:46.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:46.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-17T05:49:46.336 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705671 2026-02-17T05:49:46.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T05:49:46.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:49:46.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T05:49:46.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:46.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574854 2026-02-17T05:49:46.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:46.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:49:46.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574854 2026-02-17T05:49:46.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:46.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574854 2026-02-17T05:49:46.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-17T05:49:46.491 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574854 2026-02-17T05:49:46.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:46.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-17T05:49:46.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:46.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411332 2026-02-17T05:49:46.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:46.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:49:46.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411332 2026-02-17T05:49:46.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:46.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411332 2026-02-17T05:49:46.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411332' 2026-02-17T05:49:46.661 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 77309411332 2026-02-17T05:49:46.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:49:46.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-17T05:49:46.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:49:46.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:46.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:46.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:49:46.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:49:46.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:49:46.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:49:46.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:49:46.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:49:46.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:49:46.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:49:47.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:49:47.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:49:47.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:47.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:811: corrupt_and_repair_jerasure: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T05:49:47.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T05:49:47.510 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T05:49:47.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T05:49:47.712 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T05:49:47.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T05:49:47.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T05:49:47.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:49:47.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T05:49:47.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:49:47.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:49:47.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:49:47.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:49:47.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T05:49:47.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T05:49:47.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T05:49:47.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T05:49:47.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T05:49:47.898 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:49:47.898 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T05:49:48.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T05:49:48.049 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:49:48.049 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:49:48.049 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T05:49:48.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 0 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('1' '2' '0') 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=1 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=2 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:49:48.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:49:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:49:49.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:49:50.009 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:49:50.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:49:50.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:49:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:49:50.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:49:50.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:49:50.567 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:50.552+0000 7fc145ded8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:50.568 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:50.556+0000 7fc145ded8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:50.569 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:50.556+0000 7fc145ded8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:49:50.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:50.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:50.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:51.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:51.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:51.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:49:51.853 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:49:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:52.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:52.049 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:52.036+0000 7fc145ded8c0 -1 Falling back to public interface 2026-02-17T05:49:53.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:53.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:53.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:49:53.006 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:49:53.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:53.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:53.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:54.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:54.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:54.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:49:54.161 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:49:54.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:54.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:54.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:49:55.051 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:49:55.036+0000 7fc145ded8c0 -1 osd.3 35 log_to_monitors true 2026-02-17T05:49:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:49:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:49:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:49:55.311 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:49:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:49:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [18,35) [v2:127.0.0.1:6826/856724409,v1:127.0.0.1:6827/856724409] [v2:127.0.0.1:6828/856724409,v1:127.0.0.1:6829/856724409] exists,up da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:49:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:49:55.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:49:55.462 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:49:55.462 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:49:55.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:49:55.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:49:55.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:49:55.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:49:55.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:49:55.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:49:55.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:49:55.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:49:55.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:49:55.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:49:55.720 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:49:55.720 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:49:55.720 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:49:55.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:49:55.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:55.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:49:55.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T05:49:55.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T05:49:55.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T05:49:55.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:55.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:49:55.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T05:49:55.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T05:49:55.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705675' 2026-02-17T05:49:55.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:55.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:49:55.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-17T05:49:55.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-17T05:49:55.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574857' 2026-02-17T05:49:55.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:49:55.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:49:56.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724546 2026-02-17T05:49:56.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724546 2026-02-17T05:49:56.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574857 3-167503724546' 2026-02-17T05:49:56.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:56.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T05:49:56.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:56.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:49:56.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T05:49:56.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:56.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T05:49:56.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T05:49:56.008 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836492 2026-02-17T05:49:56.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:49:56.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836492 2026-02-17T05:49:56.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:49:57.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:49:57.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:49:57.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836492 2026-02-17T05:49:57.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:57.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T05:49:57.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:57.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:49:57.328 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T05:49:57.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:57.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T05:49:57.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T05:49:57.330 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705675 2026-02-17T05:49:57.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:49:57.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T05:49:57.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:57.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-17T05:49:57.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:57.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:49:57.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-17T05:49:57.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:57.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-17T05:49:57.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T05:49:57.497 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 55834574857 2026-02-17T05:49:57.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:49:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-17T05:49:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:49:57.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-167503724546 2026-02-17T05:49:57.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:49:57.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:49:57.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-167503724546 2026-02-17T05:49:57.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:49:57.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724546 2026-02-17T05:49:57.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 167503724546' 2026-02-17T05:49:57.650 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 167503724546 2026-02-17T05:49:57.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:49:57.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-17T05:49:57.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:49:57.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:57.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:57.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:49:57.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:49:57.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:49:57.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:49:57.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:49:57.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:49:57.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:49:57.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:49:58.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:49:58.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:49:58.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:49:58.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:49:58.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:49:58.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:49:58.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:49:58.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:49:58.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:49:58.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:49:58.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:49:58.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:49:58.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:49:58.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:49:58.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:49:58.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:49:58.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:49:58.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:49:58.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:49:58.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:49:58.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:49:43.207503+0000 2026-02-17T05:49:58.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:49:58.754 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:49:58.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:49:43.207503+0000 2026-02-17T05:49:58.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:49:58.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:49:43.207503+0000 2026-02-17T05:49:58.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:49:58.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:49:58.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:49:58.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:49:58.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:49:58.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:49:58.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:49:58.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:49:58.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:43.207503+0000 '>' 2026-02-17T05:49:43.207503+0000 2026-02-17T05:49:58.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:49:59.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:49:59.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:49:59.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:49:59.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:49:59.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:49:59.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:49:59.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:00.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:43.207503+0000 '>' 2026-02-17T05:49:43.207503+0000 2026-02-17T05:50:00.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:01.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:01.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:01.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:01.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:01.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:01.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:01.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:01.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:43.207503+0000 '>' 2026-02-17T05:49:43.207503+0000 2026-02-17T05:50:01.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:02.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:02.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:02.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:02.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:02.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:02.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:02.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:02.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:59.114583+0000 '>' 2026-02-17T05:49:43.207503+0000 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:50:02.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:50:02.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:50:02.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:50:02.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:50:02.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:50:02.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:50:02.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:50:03.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:50:03.997 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:50:03.997 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:50:03.997 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:50:03.997 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:50:04.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:50:04.535 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:50:04.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:50:04.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:50:04.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:50:04.555 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:04.540+0000 7ff41ca678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:04.556 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:04.544+0000 7ff41ca678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:04.557 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:04.544+0000 7ff41ca678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:50:04.692 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:50:04.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:04.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:04.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:05.782 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:05.768+0000 7ff41ca678c0 -1 Falling back to public interface 2026-02-17T05:50:05.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:05.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:05.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:50:05.844 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:50:05.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:05.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:05.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:06.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:06.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:06.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:50:06.992 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:50:06.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:06.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:07.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:50:08.140 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:50:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:08.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:08.761 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:08.747+0000 7ff41ca678c0 -1 osd.3 40 log_to_monitors true 2026-02-17T05:50:09.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:09.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:09.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:50:09.291 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:50:09.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:09.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:09.436 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 44 up_thru 44 down_at 41 last_clean_interval [39,40) [v2:127.0.0.1:6826/1988474833,v1:127.0.0.1:6827/1988474833] [v2:127.0.0.1:6828/1988474833,v1:127.0.0.1:6829/1988474833] exists,up da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:50:09.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:50:09.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:50:09.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:50:09.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:50:09.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:50:09.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:50:09.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:50:09.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:50:09.438 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:50:09.438 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:50:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:50:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:50:09.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:50:09.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:50:09.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:50:09.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:50:09.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:50:09.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:50:09.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:50:09.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:50:09.703 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:50:09.703 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:50:09.703 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:50:09.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:50:09.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:09.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:50:09.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836496 2026-02-17T05:50:09.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836496 2026-02-17T05:50:09.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496' 2026-02-17T05:50:09.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:09.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:50:09.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705678 2026-02-17T05:50:09.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705678 2026-02-17T05:50:09.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-38654705678' 2026-02-17T05:50:09.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:09.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:50:09.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574861 2026-02-17T05:50:09.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574861 2026-02-17T05:50:09.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-38654705678 2-55834574861' 2026-02-17T05:50:09.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:09.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:50:09.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T05:50:09.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T05:50:09.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-38654705678 2-55834574861 3-188978561026' 2026-02-17T05:50:09.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:09.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836496 2026-02-17T05:50:09.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:09.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:50:09.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836496 2026-02-17T05:50:09.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:09.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836496 2026-02-17T05:50:09.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-17T05:50:09.995 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836496 2026-02-17T05:50:09.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:10.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-17T05:50:10.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:11.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:50:11.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:11.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-17T05:50:11.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:12.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:50:12.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:12.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-17T05:50:12.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:12.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705678 2026-02-17T05:50:12.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:12.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:50:12.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705678 2026-02-17T05:50:12.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:12.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705678 2026-02-17T05:50:12.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705678' 2026-02-17T05:50:12.458 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705678 2026-02-17T05:50:12.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:50:12.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705679 -lt 38654705678 2026-02-17T05:50:12.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:12.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574861 2026-02-17T05:50:12.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:12.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:50:12.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574861 2026-02-17T05:50:12.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:12.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574861 2026-02-17T05:50:12.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574861' 2026-02-17T05:50:12.612 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 55834574861 2026-02-17T05:50:12.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:50:12.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574861 -lt 55834574861 2026-02-17T05:50:12.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:12.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-188978561026 2026-02-17T05:50:12.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:12.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:50:12.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-188978561026 2026-02-17T05:50:12.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T05:50:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 188978561026' 2026-02-17T05:50:12.763 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 188978561026 2026-02-17T05:50:12.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:50:12.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T05:50:12.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:50:12.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:12.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:13.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:50:13.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:50:13.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:50:13.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:50:13.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:50:13.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:50:13.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:50:13.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:50:13.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:50:13.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:50:13.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:13.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:13.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:50:13.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:50:13.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:50:13.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:50:13.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:50:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 1 2026-02-17T05:50:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:50:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:50:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=1 2026-02-17T05:50:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:50:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:50:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:50:13.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:50:13.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:50:13.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:50:14.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:50:14.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:50:14.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:50:14.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:50:14.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:50:14.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:50:14.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:50:14.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:50:15.395 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:50:15.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:50:15.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:50:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:50:15.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:50:15.933 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:50:15.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:50:15.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:50:15.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:50:15.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:50:15.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:50:15.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:50:15.948 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:15.935+0000 7fd258e868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:15.950 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:15.935+0000 7fd258e868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:15.951 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:15.935+0000 7fd258e868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:50:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:50:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:50:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:50:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:50:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:16.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:50:16.090 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:50:16.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:16.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:16.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:17.194 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:17.179+0000 7fd258e868c0 -1 Falling back to public interface 2026-02-17T05:50:17.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:17.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:17.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:50:17.241 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:50:17.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:17.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:17.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:18.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:18.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:18.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:50:18.391 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:50:18.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:18.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:18.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:19.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:19.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:19.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:50:19.544 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:50:19.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:19.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:19.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:20.202 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:20.187+0000 7fd258e868c0 -1 osd.1 46 log_to_monitors true 2026-02-17T05:50:20.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:20.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:20.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:50:20.695 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:50:20.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:20.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 47 last_clean_interval [9,46) [v2:127.0.0.1:6810/220699825,v1:127.0.0.1:6811/220699825] [v2:127.0.0.1:6812/220699825,v1:127.0.0.1:6813/220699825] exists,up 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:50:20.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:50:20.842 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:50:20.842 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:50:20.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:50:20.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:50:20.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:50:20.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:50:20.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:50:20.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:50:20.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:50:20.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:50:20.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:50:21.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:50:21.101 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:50:21.101 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:50:21.101 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:50:21.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:50:21.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:21.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:50:21.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836499 2026-02-17T05:50:21.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836499 2026-02-17T05:50:21.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499' 2026-02-17T05:50:21.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:21.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:50:21.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=214748364802 2026-02-17T05:50:21.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 214748364802 2026-02-17T05:50:21.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-214748364802' 2026-02-17T05:50:21.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:21.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:50:21.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574864 2026-02-17T05:50:21.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574864 2026-02-17T05:50:21.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-214748364802 2-55834574864' 2026-02-17T05:50:21.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:21.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:50:21.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561029 2026-02-17T05:50:21.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561029 2026-02-17T05:50:21.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-214748364802 2-55834574864 3-188978561029' 2026-02-17T05:50:21.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:21.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836499 2026-02-17T05:50:21.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:21.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:50:21.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836499 2026-02-17T05:50:21.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:21.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836499 2026-02-17T05:50:21.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-17T05:50:21.391 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836499 2026-02-17T05:50:21.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:21.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-17T05:50:21.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:21.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-214748364802 2026-02-17T05:50:21.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:21.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:50:21.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-214748364802 2026-02-17T05:50:21.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:21.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=214748364802 2026-02-17T05:50:21.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 214748364802' 2026-02-17T05:50:21.556 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 214748364802 2026-02-17T05:50:21.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:50:21.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 214748364802 -lt 214748364802 2026-02-17T05:50:21.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:21.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574864 2026-02-17T05:50:21.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:21.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:50:21.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574864 2026-02-17T05:50:21.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:21.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574864 2026-02-17T05:50:21.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-17T05:50:21.718 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 55834574864 2026-02-17T05:50:21.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:50:21.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574863 -lt 55834574864 2026-02-17T05:50:21.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:22.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:50:22.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:50:23.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574863 -lt 55834574864 2026-02-17T05:50:23.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:24.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:50:24.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:50:24.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-17T05:50:24.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:24.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-188978561029 2026-02-17T05:50:24.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:24.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:50:24.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-188978561029 2026-02-17T05:50:24.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:24.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561029 2026-02-17T05:50:24.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 188978561029' 2026-02-17T05:50:24.191 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 188978561029 2026-02-17T05:50:24.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:50:24.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 188978561029 2026-02-17T05:50:24.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:50:24.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:24.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:50:24.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:50:24.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:50:24.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:50:24.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:24.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:24.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:50:24.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:50:24.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:50:24.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:50:24.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:50:24.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:50:24.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:50:24.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:50:25.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:50:25.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:50:25.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:50:25.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:50:25.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:25.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:25.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:25.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:25.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:25.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:50:25.325 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:50:25.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:25.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:25.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:25.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:59.114583+0000 '>' 2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:25.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:26.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:26.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:26.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:26.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:26.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:26.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:26.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:26.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:59.114583+0000 '>' 2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:26.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:27.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:27.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:27.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:27.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:27.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:27.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:27.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:27.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:59.114583+0000 '>' 2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:27.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:28.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:28.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:28.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:28.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:28.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:28.799 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:28.799 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:28.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:49:59.114583+0000 '>' 2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:28.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:29.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:29.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:29.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:29.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:29.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:29.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:29.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:30.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:25.658091+0000 '>' 2026-02-17T05:49:59.114583+0000 2026-02-17T05:50:30.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:50:30.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:50:30.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:50:31.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:50:31.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:50:31.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:50:31.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:50:31.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:50:31.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:50:31.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:50:31.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:50:31.719 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:50:31.719 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:50:31.719 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:50:31.719 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:50:32.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:50:32.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:50:32.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:50:32.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:50:32.262 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:50:32.262 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:50:32.262 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:50:32.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:50:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:50:32.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:50:32.264 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:50:32.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:50:32.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:50:32.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:50:32.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:50:32.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:50:32.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:50:32.288 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:32.275+0000 7f000397e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:32.290 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:32.275+0000 7f000397e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:32.291 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:32.275+0000 7f000397e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:32.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:50:32.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:32.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:32.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:33.518 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:33.503+0000 7f000397e8c0 -1 Falling back to public interface 2026-02-17T05:50:33.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:33.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:33.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:50:33.577 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:50:33.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:33.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:33.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:34.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:34.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:34.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:50:34.728 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:50:34.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:34.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:34.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:35.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:35.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:35.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:50:35.880 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:50:35.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:35.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:36.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:36.513 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:50:36.499+0000 7f000397e8c0 -1 osd.3 53 log_to_monitors true 2026-02-17T05:50:37.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:37.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:37.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:50:37.037 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:50:37.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:37.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:50:37.191 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 57 up_thru 57 down_at 54 last_clean_interval [44,53) [v2:127.0.0.1:6826/846628202,v1:127.0.0.1:6827/846628202] [v2:127.0.0.1:6828/846628202,v1:127.0.0.1:6829/846628202] exists,up da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:50:37.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:50:37.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:50:37.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:50:37.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:50:37.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:50:37.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:50:37.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:50:37.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:50:37.192 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:50:37.192 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:50:37.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:50:37.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:50:37.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:50:37.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:50:37.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:50:37.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:50:37.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:50:37.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:50:37.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:50:37.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:50:37.441 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:50:37.441 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:50:37.442 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T05:50:37.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:50:37.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:37.442 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:50:37.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836504 2026-02-17T05:50:37.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836504 2026-02-17T05:50:37.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504' 2026-02-17T05:50:37.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:37.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:50:37.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=214748364806 2026-02-17T05:50:37.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 214748364806 2026-02-17T05:50:37.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-214748364806' 2026-02-17T05:50:37.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:37.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:50:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574868 2026-02-17T05:50:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574868 2026-02-17T05:50:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-214748364806 2-55834574868' 2026-02-17T05:50:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:37.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:50:37.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135874 2026-02-17T05:50:37.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135874 2026-02-17T05:50:37.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-214748364806 2-55834574868 3-244813135874' 2026-02-17T05:50:37.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:37.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836504 2026-02-17T05:50:37.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:37.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:50:37.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836504 2026-02-17T05:50:37.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:37.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836504 2026-02-17T05:50:37.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-17T05:50:37.735 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836504 2026-02-17T05:50:37.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:37.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836503 -lt 21474836504 2026-02-17T05:50:37.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:38.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:50:38.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:39.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836503 -lt 21474836504 2026-02-17T05:50:39.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:40.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:50:40.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:40.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836504 -lt 21474836504 2026-02-17T05:50:40.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:40.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-214748364806 2026-02-17T05:50:40.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:40.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:50:40.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-214748364806 2026-02-17T05:50:40.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:40.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=214748364806 2026-02-17T05:50:40.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 214748364806' 2026-02-17T05:50:40.208 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 214748364806 2026-02-17T05:50:40.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:50:40.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 214748364806 -lt 214748364806 2026-02-17T05:50:40.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:40.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574868 2026-02-17T05:50:40.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:40.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:50:40.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574868 2026-02-17T05:50:40.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574868 2026-02-17T05:50:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574868' 2026-02-17T05:50:40.372 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 55834574868 2026-02-17T05:50:40.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:50:40.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574869 -lt 55834574868 2026-02-17T05:50:40.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:40.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135874 2026-02-17T05:50:40.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:40.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:50:40.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135874 2026-02-17T05:50:40.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:40.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135874 2026-02-17T05:50:40.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135874' 2026-02-17T05:50:40.531 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 244813135874 2026-02-17T05:50:40.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:50:40.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135874 -lt 244813135874 2026-02-17T05:50:40.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:50:40.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:40.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:50:40.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:50:41.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:50:41.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:50:41.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:41.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:41.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:50:41.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:50:41.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:50:41.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:50:41.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 1 2 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=1 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=2 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 158061"' 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 158061' 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:50:41.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 158062"' 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 158062' 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 158061 158062' 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 158061 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/158064: /' 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:50:41.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/158066: /' 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:50:43.714 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:50:43.717 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T05:50:43.718 INFO:tasks.workunit.client.0.trial074.stderr:158066: start osd.2 2026-02-17T05:50:43.718 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:50:43.984 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.984 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:50:43.984 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:50:43.985 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:50:43.987 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:50:43.987 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:50:43.987 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: start osd.1 2026-02-17T05:50:43.988 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:50:48.284 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: 2026-02-17T05:50:43.719+0000 7f588d1648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: 2026-02-17T05:50:43.723+0000 7f588d1648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: 2026-02-17T05:50:43.723+0000 7f588d1648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: 0 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: 1 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: 2026-02-17T05:50:45.259+0000 7f588d1648c0 -1 Falling back to public interface 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:48.285 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: 2 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: 3 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:50:48.286 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158066: 2026-02-17T05:50:48.271+0000 7f588d1648c0 158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: 2026-02-17T05:50:43.987+0000 7f010a35e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: 2026-02-17T05:50:43.987+0000 7f010a35e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: 2026-02-17T05:50:43.991+0000 7f010a35e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:50:48.511 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: 0 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:50:48.512 INFO:tasks.workunit.client.0.trial074.stderr:158064: 1 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: 2026-02-17T05:50:45.451+0000 7f010a35e8c0 -1 Falling back to public interface 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: 2 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.513 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:48.514 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.514 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:50:48.514 INFO:tasks.workunit.client.0.trial074.stderr:158064: 3 2026-02-17T05:50:48.514 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.514 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:48.514 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:50:48.971 INFO:tasks.workunit.client.0.trial074.stderr:158064: 2026-02-17T05:50:48.495+0000 7f010a35e8c0 -1 osd.2 59 log_to_monitors true 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: 4 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: osd.2 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [13,59) [v2:127.0.0.1:6810/1529029385,v1:127.0.0.1:6811/1529029385] [v2:127.0.0.1:6812/1529029385,v1:127.0.0.1:6813/1529029385] exists,up 61db70a1-40d3-428d-851a-55fbe9fddac6 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: 1 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: 2 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: 3' 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836507 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836507 2026-02-17T05:50:48.972 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507' 2026-02-17T05:50:48.973 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:49.239 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel-1 osd.1 59 log_to_monitors true 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: 4 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: osd.1 up in weight 1 up_from 65 up_thru 65 down_at 60 last_clean_interval [50,59) [v2:127.0.0.1:6818/1780769415,v1:127.0.0.1:6819/1780769415] [v2:127.0.0.1:6820/1780769415,v1:127.0.0.1:6821/1780769415] exists,up 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:50:49.240 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:50:49.241 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: 1 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: 2 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: 3' 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836508 2026-02-17T05:50:49.242 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836508 2026-02-17T05:50:49.243 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508' 2026-02-17T05:50:49.243 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:49.497 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:50:49.497 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874242 2026-02-17T05:50:49.497 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874242 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-279172874242' 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939650 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939650 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-279172874242 2-270582939650' 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135877 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135877 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-279172874242 2-270582939650 3-244813135877' 2026-02-17T05:50:49.498 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836507 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836507 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836507 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: waiting osd.0 seq 21474836507 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836507 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:49.499 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-279172874242 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-279172874242 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874242 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 279172874242' 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: waiting osd.1 seq 279172874242 2026-02-17T05:50:49.500 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:50:49.761 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:227pers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:50:49.761 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874243 2026-02-17T05:50:49.761 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874243 2026-02-17T05:50:49.761 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-279172874243' 2026-02-17T05:50:49.761 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939651 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939651 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-279172874243 2-270582939651' 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135878 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135878 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-279172874243 2-270582939651 3-244813135878' 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836508 2026-02-17T05:50:49.762 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836508 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836508 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: waiting osd.0 seq 21474836508 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-279172874243 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:50:49.763 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-279172874243 2026-02-17T05:50:49.764 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:49.764 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874243 2026-02-17T05:50:49.764 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 279172874243' 2026-02-17T05:50:49.764 INFO:tasks.workunit.client.0.trial074.stderr:158064: waiting osd.1 seq 279172874243 2026-02-17T05:50:49.764 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:50:50.151 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874242 -lt 279172874242 2026-02-17T05:50:50.151 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:50.151 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-270582939650 2026-02-17T05:50:50.151 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:50.151 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:50:50.151 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-270582939650 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939650 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 270582939650' 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: waiting osd.2 seq 270582939650 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 270582939650 -lt 270582939650 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135877 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135877 2026-02-17T05:50:50.152 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135877 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135877' 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: waiting osd.3 seq 244813135877 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135877 -lt 244813135877 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:50:50.153 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:50:50.154 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:50:50.154 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:50:50.154 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:50:50.154 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:158066: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:158066: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:50:50.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:7: flush_pg_stats: test 279172874242 -lt 279172874243 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874242 -lt 279172874243 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:50:52.585 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874243 -lt 279172874243 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-270582939651 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-270582939651 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939651 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 270582939651' 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: waiting osd.2 seq 270582939651 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:50:52.586 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135878 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135878 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135878 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135878' 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: waiting osd.3 seq 244813135878 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135878 -lt 244813135878 2026-02-17T05:50:52.587 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:50:52.588 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:52.588 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:52.588 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:50:52.588 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:50:52.913 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:50:52.914 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:158064: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 158062 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:50:52.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:50:52.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:50:52.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:50:52.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:50:53.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:50:53.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:50:53.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:50:53.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:50:53.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:53.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:53.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:53.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:53.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:53.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:50:53.352 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:50:53.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:53.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:50:53.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:53.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:50:53.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:50:53.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:53.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:53.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:53.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:53.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:53.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:53.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:25.658091+0000 '>' 2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:53.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:54.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:54.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:54.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:54.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:54.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:54.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:54.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:54.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:25.658091+0000 '>' 2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:54.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:55.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:55.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:55.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:55.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:55.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:55.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:55.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:55.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:25.658091+0000 '>' 2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:55.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:56.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:56.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:56.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:56.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:56.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:56.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:56.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:56.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:25.658091+0000 '>' 2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:56.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:50:57.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:50:57.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:50:57.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:50:57.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:50:57.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:50:57.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:50:57.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:53.359530+0000 '>' 2026-02-17T05:50:25.658091+0000 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 161017"' 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 161017' 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:50:58.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 161019"' 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 161019' 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 161017 161019' 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 161017 2026-02-17T05:50:58.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/161020: /' 2026-02-17T05:50:58.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:50:58.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/161022: /' 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T05:51:00.297 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING list-attrs 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: _ 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: hinfo_key 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: snapset 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:00.298 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: _ 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: hinfo_key 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: snapset 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:00.299 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:00.300 INFO:tasks.workunit.client.0.trial074.stderr:161022: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: start osd.2 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recs.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:00.301 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:51:00.302 INFO:tasks.workunit.client.0.trial074.stderr:161020: start osd.1 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: 2026-02-17T05:51:00.299+0000 7f40b687e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:04.600 INFO:tasks.workunit.client.0.trial074.stderr:161020: 2026-02-17T05:51:00.303+0000 7f40b687e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: 2026-02-17T05:51:00.303+0000 7f40b687e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: 0 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:04.601 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: 1 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: 2026-02-17T05:51:01.599+0000 7f40b687e8c0 -1 Falling back to public interface 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:04.602 INFO:tasks.workunit.client.0.trial074.stderr:161020: 2 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: 3 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:04.603 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:04.839 INFO:tasks.workunit.client.0.trial074.stderr:161020: 2026-02-17T05:51:04.583+0000 7f4overy_ops 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: 2026-02-17T05:51:00.299+0000 7f15ebd6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: 2026-02-17T05:51:00.303+0000 7f15ebd6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: 2026-02-17T05:51:00.303+0000 7f15ebd6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T05:51:04.840 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: 0 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:04.841 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: 1 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: 2026-02-17T05:51:01.767+0000 7f15ebd6a8c0 -1 Falling back to public interface 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: 2 2026-02-17T05:51:04.842 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: 3 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:51:04.843 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:05.547 INFO:tasks.workunit.client.0.trial074.stderr:161022: 2026-02-17T05:51:04.823+0000 7f10b687e8c0 -1 osd.1 67 log_to_monitors true 2026-02-17T05:51:05.547 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:05.547 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:05.547 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:05.547 INFO:tasks.workunit.client.0.trial074.stderr:161020: 4 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: osd.1 up in weight 1 up_from 71 up_thru 73 down_at 68 last_clean_interval [65,67) [v2:127.0.0.1:6810/4279198494,v1:127.0.0.1:6811/4279198494] [v2:127.0.0.1:6812/4279198494,v1:127.0.0.1:6813/4279198494] exists,up 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:51:05.548 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:51:05.549 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: 1 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: 2 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: 3' 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836512 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836512 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512' 2026-02-17T05:51:05.550 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalon5ebd6a8c0 -1 osd.2 67 log_to_monitors true 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161022: 4 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:51:05.556 INFO:tasks.workunit.client.0.trial074.stderr:161022: osd.2 up in weight 1 up_from 73 up_thru 63 down_at 68 last_clean_interval [63,67) [v2:127.0.0.1:6818/1274236403,v1:127.0.0.1:6819/1274236403] [v2:127.0.0.1:6820/1274236403,v1:127.0.0.1:6821/1274236403] exists,up 61db70a1-40d3-428d-851a-55fbe9fddac6 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:51:05.557 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: 1 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: 2 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: 3' 2026-02-17T05:51:05.558 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:51:05.559 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:05.559 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:51:05.559 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836513 2026-02-17T05:51:05.559 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836513 2026-02-17T05:51:05.559 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513' 2026-02-17T05:51:05.559 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:08.225 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:51:08.225 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678018 2026-02-17T05:51:08.225 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678018 2026-02-17T05:51:08.226 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-304942678018' 2026-02-17T05:51:08.226 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:08.226 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:51:08.226 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=313532612610 2026-02-17T05:51:08.226 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 313532612610 2026-02-17T05:51:08.226 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-304942678018 2-313532612610' 2026-02-17T05:51:08.226 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135882 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135882 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-304942678018 2-313532612610 3-244813135882' 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836512 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:51:08.227 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836512 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836512 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836512' 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: waiting osd.0 seq 21474836512 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836512 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836512 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:08.228 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:51:08.229 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:08.229 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836514 -lt 21474836512 2026-02-17T05:51:08.229 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_statse/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678019 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678019 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-304942678019' 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=313532612611 2026-02-17T05:51:08.230 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 313532612611 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-304942678019 2-313532612611' 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135883 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135883 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-304942678019 2-313532612611 3-244813135883' 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836513 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836513 2026-02-17T05:51:08.231 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836513 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: waiting osd.0 seq 21474836513 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836513 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836513 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:51:08.232 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:08.233 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836514 -lt 21474836513 2026-02-17T05:51:08.233 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-304942678018 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-304942678018 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678018 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 304942678018' 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: waiting osd.1 seq 304942678018 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678019 -lt 304942678018 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-313532612610 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.888 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-313532612610 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=313532612610 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 313532612610' 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: waiting osd.2 seq 313532612610 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612611 -lt 313532612610 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135882 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135882 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135882 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135882' 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: waiting osd.3 seq 244813135882 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135884 -lt 244813135882 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:08.889 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/c: echo 1-304942678019 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-304942678019 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678019 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 304942678019' 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: waiting osd.1 seq 304942678019 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678019 -lt 304942678019 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-313532612611 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-313532612611 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=313532612611 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 313532612611' 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: waiting osd.2 seq 313532612611 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612611 -lt 313532612611 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135883 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135883 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135883 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135883' 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: waiting osd.3 seq 244813135883 2026-02-17T05:51:08.890 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:51:08.891 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135884 -lt 244813135883 2026-02-17T05:51:08.891 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:51:08.891 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:08.891 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:08.891 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:51:08.891 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:09.657 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:09.658 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:09.658 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:09.658 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:09.658 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:09.658 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:09.658 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:09.658 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' -1 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:09.659 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:09.660 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:09.661 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:51:09.661 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:51:09.661 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:51:09.668 INFO:tasks.workunit.client.0.trial074.stderr:161020: /homeeph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:09.668 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:09.669 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:09.670 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:09.670 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' -1 2026-02-17T05:51:09.670 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:51:09.670 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T05:51:09.670 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:09.670 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:51:09.671 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:09.671 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:09.671 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:09.671 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:09.671 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:09.671 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:09.672 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:09.672 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:09.672 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:09.672 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:09.672 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:09.672 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:09.672 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:09.673 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:51:09.673 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:51:09.673 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:10.377 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:51:10.378 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:51:10.384 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:51:10.385 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:51:10.385 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:51:10.385 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:51:10.385 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:51:11.484 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:51:11.484 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:51:11.484 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:51:11.484 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:51:11.484 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:11.485 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:51:11.486 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:51:11.501 INFO:tasks.workunit.client.0.trial074.stderr:161020: /h: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:51:11.501 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:51:11.501 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:51:11.501 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T05:51:11.502 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:51:11.503 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:51:11.504 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:51:11.504 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:51:11.504 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:12.631 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161022: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:ome/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:51:12.632 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:12.633 INFO:tasks.workunit.client.0.trial074.stderr:161020: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:161020: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 161019 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:51:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:51:12.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:51:12.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 1 2026-02-17T05:51:12.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:51:12.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:51:12.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T05:51:12.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=1 2026-02-17T05:51:12.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:51:12.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:51:12.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:51:12.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:51:12.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 164016"' 2026-02-17T05:51:12.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 164016' 2026-02-17T05:51:12.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:51:12.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:51:12.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:51:12.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 164018"' 2026-02-17T05:51:12.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 164018' 2026-02-17T05:51:12.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:51:12.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:51:12.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 164016 164018' 2026-02-17T05:51:12.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:51:12.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:51:12.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 164016 2026-02-17T05:51:12.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/164019: /' 2026-02-17T05:51:12.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:51:12.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/164021: /' 2026-02-17T05:51:15.130 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.130 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:51:15.130 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:51:15.130 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:51:15.130 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:51:15.130 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:15.131 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:51:15.132 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:51:15.133 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:51:15.133 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:15.133 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:15.133 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:15.134 INFO:tasks.workunit.client.0.trial074.stderr:164021: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:15.135 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:51:15.136 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:51:15.136 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:51:15.136 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:51:15.136 INFO:tasks.workunit.client.0.trial074.stderr:164021: start osd.1 2026-02-17T05:51:15.136 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:51:15.164 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.164 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:15.165 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:51:15.166 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:51:15.167 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:15.167 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:51:15.167 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:51:15.167 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:51:15.167 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:51:15.167 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:15.167 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:15.168 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:15.169 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:15.170 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:15.170 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:51:15.170 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:51:15.170 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:51:15.170 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:51:15.170 INFO:tasks.workunit.client.0.trial074.stderr:164019: start osd.3 2026-02-17T05:51:15.170 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:51:19.467 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:51:19.467 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: 2026-02-17T05:51:15.143+0000 7ffa996588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: 2026-02-17T05:51:15.143+0000 7ffa996588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: 2026-02-17T05:51:15.143+0000 7ffa996588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:19.468 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: 0 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: 2026-02-17T05:51:16.419+0000 7ffa996588c0 -1 Falling back to public interface 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: 1 2026-02-17T05:51:19.469 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: 2 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.470 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:19.471 INFO:tasks.workunit.client.0.trial074.stderr:164021: 3 2026-02-17T05:51:19.471 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.471 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:19.471 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:19.732 INFO:tasks.workunit.client.0.trial074.stderr:164021: 2026-02-17T05:51:19.451+0000 7ffa996588c0 164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:51:19.732 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:51:19.732 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:51:19.732 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:51:19.732 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:51:19.732 INFO:tasks.workunit.client.0.trial074.stderr:164019: 2026-02-17T05:51:15.175+0000 7fa33b9cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:19.732 INFO:tasks.workunit.client.0.trial074.stderr:164019: 2026-02-17T05:51:15.179+0000 7fa33b9cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: 2026-02-17T05:51:15.183+0000 7fa33b9cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: 0 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:19.733 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: 1 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: 2026-02-17T05:51:16.691+0000 7fa33b9cc8c0 -1 Falling back to public interface 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:19.734 INFO:tasks.workunit.client.0.trial074.stderr:164019: 2 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: 3 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:19.735 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:20.395 INFO:tasks.workunit.client.0.trial074.stderr:164019: 2026-02-17T05:51:19.719+0000 7fa33b9cc8c0 -1 osd.1 74 log_to_monitors true 2026-02-17T05:51:20.395 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:20.395 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:20.395 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:20.395 INFO:tasks.workunit.client.0.trial074.stderr:164021: 4 2026-02-17T05:51:20.395 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: osd.1 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [71,74) [v2:127.0.0.1:6810/1743126208,v1:127.0.0.1:6811/1743126208] [v2:127.0.0.1:6812/1743126208,v1:127.0.0.1:6813/1743126208] exists,up 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:51:20.396 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: 1 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: 2 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: 3' 2026-02-17T05:51:20.397 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:51:20.398 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:20.398 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:51:20.398 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836517 2026-02-17T05:51:20.398 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836517 2026-02-17T05:51:20.398 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517' 2026-02-17T05:51:20.398 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:20.419 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel-1 osd.3 74 log_to_monitors true 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: 4 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: osd.3 up in weight 1 up_from 81 up_thru 81 down_at 75 last_clean_interval [57,74) [v2:127.0.0.1:6826/539499241,v1:127.0.0.1:6827/539499241] [v2:127.0.0.1:6828/539499241,v1:127.0.0.1:6829/539499241] exists,up da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:20.420 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:51:20.421 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: 1 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: 2 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: 3' 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:20.422 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:51:20.423 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836518 2026-02-17T05:51:20.423 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836518 2026-02-17T05:51:20.423 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518' 2026-02-17T05:51:20.423 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helperspers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=335007449090 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 335007449090 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-335007449090' 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=313532612615 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 313532612615 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-335007449090 2-313532612615' 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=347892350978 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 347892350978 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-335007449090 2-313532612615 3-347892350978' 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836517 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836517 2026-02-17T05:51:21.933 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836517 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: waiting osd.0 seq 21474836517 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836516 -lt 21474836517 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836518 -lt 21474836517 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-335007449090 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-335007449090 2026-02-17T05:51:21.934 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/u.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=335007449091 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 335007449091 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-335007449091' 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=313532612616 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 313532612616 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-335007449091 2-313532612616' 2026-02-17T05:51:21.939 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=347892350979 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 347892350979 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-335007449091 2-313532612616 3-347892350979' 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836518 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836518 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836518 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: waiting osd.0 seq 21474836518 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836516 -lt 21474836518 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836518 -lt 21474836518 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-335007449091 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-335007449091 2026-02-17T05:51:21.940 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntbuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=335007449090 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 335007449090' 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: waiting osd.1 seq 335007449090 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 335007449091 -lt 335007449090 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-313532612615 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-313532612615 2026-02-17T05:51:22.578 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=313532612615 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 313532612615' 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: waiting osd.2 seq 313532612615 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612616 -lt 313532612615 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-347892350978 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-347892350978 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=347892350978 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 347892350978' 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: waiting osd.3 seq 347892350978 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 347892350979 -lt 347892350978 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:22.579 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:22.604 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("u/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=335007449091 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 335007449091' 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: waiting osd.1 seq 335007449091 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-313532612616 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-313532612616 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=313532612616 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 313532612616' 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: waiting osd.2 seq 313532612616 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612616 -lt 313532612616 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-347892350979 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-347892350979 2026-02-17T05:51:22.605 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=347892350979 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 347892350979' 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: waiting osd.3 seq 347892350979 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 347892350979 -lt 347892350979 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:22.606 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stalstale") | not)' 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164021: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:22.923 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:51:22.924 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:51:22.924 INFO:tasks.workunit.client.0.trial074.stderr:164021: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:51:22.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:e") | not)' 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:51:22.934 INFO:tasks.workunit.client.0.trial074.stderr:164019: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 164018 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:51:22.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:51:22.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:51:22.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:51:22.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:51:22.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:51:23.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:51:23.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:51:23.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:51:23.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:51:23.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:51:23.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:51:23.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:51:23.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:51:23.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:50:53.359530+0000 2026-02-17T05:51:23.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:51:23.383 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:51:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:50:53.359530+0000 2026-02-17T05:51:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:51:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:50:53.359530+0000 2026-02-17T05:51:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:51:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:51:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:51:23.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:51:23.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:51:23.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:51:23.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:51:23.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:51:23.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:53.359530+0000 '>' 2026-02-17T05:50:53.359530+0000 2026-02-17T05:51:23.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:51:24.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:51:24.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:51:24.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:51:24.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:51:24.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:51:24.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:51:24.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:51:24.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:50:53.359530+0000 '>' 2026-02-17T05:50:53.359530+0000 2026-02-17T05:51:24.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:51:25.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:51:25.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:51:25.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:51:25.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:51:25.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:51:25.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:51:25.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:51:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:51:23.732547+0000 '>' 2026-02-17T05:50:53.359530+0000 2026-02-17T05:51:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:51:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:51:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:51:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 166922"' 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 166922' 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 166924"' 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 166924' 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:51:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 166922 166924' 2026-02-17T05:51:25.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:51:25.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:51:25.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 166922 2026-02-17T05:51:25.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/166925: /' 2026-02-17T05:51:25.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:51:25.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/166928: /' 2026-02-17T05:51:28.009 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.009 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:51:28.009 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:51:28.010 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:51:28.010 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:51:28.010 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.010 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:51:28.010 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:51:28.010 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:51:28.010 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:51:28.011 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: _ 2026-02-17T05:51:28.012 INFO:tasks.workunit.client.0.trial074.stderr:166925: hinfo_key 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: snapset 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:51:28.013 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:28.014 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:28.014 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:51:28.015 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:28.016 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:51:28.017 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:51:28.018 INFO:tasks.workunit.client.0.trial074.stderr:166925: start osd.3 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:51:28.035 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:51:28.036 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: _ 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: hinfo_key 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: snapset 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:51:28.037 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:28.038 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:28.038 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:28.038 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:28.038 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:51:28.039 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:51:28.039 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:28.039 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:51:28.039 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:28.039 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:51:28.039 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:28.040 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:28.040 INFO:tasks.workunit.client.0.trial074.stderr:166928: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:28.040 INFO:tasks.workunit.client.0.trial074.stderr:166928: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:28.040 INFO:tasks.workunit.client.0.trial074.stderr:166928: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:28.040 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:28.040 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:28.040 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:51:28.041 INFO:tasks.workunit.client.0.trial074.stderr:166928: start osd.1 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reclen=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: 2026-02-17T05:51:28.023+0000 7fa6972688c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: 2026-02-17T05:51:28.023+0000 7fa6972688c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: 2026-02-17T05:51:28.023+0000 7fa6972688c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:31.783 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: 0 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: 2026-02-17T05:51:29.295+0000 7fa6972688c0 -1 Falling back to public interface 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: 1 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: 2 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:31.784 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:31.785 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:31.785 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:31.785 INFO:tasks.workunit.client.0.trial074.stderr:166925: 3 2026-02-17T05:51:31.785 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:31.785 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: 2026-02-17T05:51:28.039+0000 7f1e848988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: 2026-02-17T05:51:28.039+0000 7f1e848988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: 2026-02-17T05:51:28.039+0000 7f1e848988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:32.397 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: 0 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: 1 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: 2026-02-17T05:51:29.331+0000 7f1e848988c0 -1 Falling back to public interface 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: 2 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:32.398 INFO:tasks.workunit.client.0.trial074.stderr:166928: 3 2026-02-17T05:51:32.399 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:32.399 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:32.399 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:33.265 INFO:tasks.workunit.client.0.trial074.stderr:166928: 2026-02-17T05:51:32.383+0000 7f1.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:33.265 INFO:tasks.workunit.client.0.trial074.stderr:166925: 2026-02-17T05:51:32.299+0000 7fa6972688c0 -1 osd.3 83 log_to_monitors true 2026-02-17T05:51:33.265 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:33.265 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:33.265 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:33.265 INFO:tasks.workunit.client.0.trial074.stderr:166925: 4 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: osd.3 up in weight 1 up_from 87 up_thru 88 down_at 84 last_clean_interval [81,83) [v2:127.0.0.1:6810/2109139040,v1:127.0.0.1:6811/2109139040] [v2:127.0.0.1:6812/2109139040,v1:127.0.0.1:6813/2109139040] exists,up da21098d-9141-419d-b0df-c787e2d8eb35 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: 1 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: 2 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: 3' 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836522 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836522 2026-02-17T05:51:33.266 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522' 2026-02-17T05:51:33.283 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helperse848988c0 -1 osd.1 83 log_to_monitors true 2026-02-17T05:51:33.283 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:33.283 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:33.283 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:33.283 INFO:tasks.workunit.client.0.trial074.stderr:166928: 4 2026-02-17T05:51:33.283 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:33.283 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: osd.1 up in weight 1 up_from 88 up_thru 88 down_at 84 last_clean_interval [78,83) [v2:127.0.0.1:6826/2202036838,v1:127.0.0.1:6827/2202036838] [v2:127.0.0.1:6828/2202036838,v1:127.0.0.1:6829/2202036838] exists,up 0a05e72b-9b3e-4b52-b609-778f78e1e2e6 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: 1 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: 2 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: 3' 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836523 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836523 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523' 2026-02-17T05:51:33.284 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:33.663 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalon.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122050 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122050 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-377957122050' 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=313532612620 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 313532612620 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-377957122050 2-313532612620' 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:51:33.664 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154755 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154755 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-377957122050 2-313532612620 3-373662154755' 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836522 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836522 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836522 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836522' 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: waiting osd.0 seq 21474836522 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836522 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:33.665 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122050 2026-02-17T05:51:33.666 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:33.666 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:51:33.666 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122050 2026-02-17T05:51:33.666 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:33.666 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122050 2026-02-17T05:51:33.666 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122050' 2026-02-17T05:51:33.666 INFO:tasks.workunit.client.0.trial074.stderr:166925: waiting osd.1 seq 377957122050 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_e/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122051 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122051 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-377957122051' 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=313532612619 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 313532612619 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-377957122051 2-313532612619' 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154754 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154754 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-377957122051 2-313532612619 3-373662154754' 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836523 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836523 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836523 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836523' 2026-02-17T05:51:35.965 INFO:tasks.workunit.client.0.trial074.stderr:166928: waiting osd.0 seq 21474836523 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836523 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836523 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836523 -lt 21474836523 2026-02-17T05:51:35.966 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:36.613 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122051 2026-02-17T05:51:36.613 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:36.613 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:51:36.613 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122051 2026-02-17T05:51:36.613 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:36.613 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122051 2026-02-17T05:51:36.613 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122051' 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: waiting osd.1 seq 377957122051 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122051 -lt 377957122051 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-313532612619 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-313532612619 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=313532612619 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 313532612619' 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: waiting osd.2 seq 313532612619 2026-02-17T05:51:36.614 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612621 -lt 313532612619 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-373662154754 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-373662154754 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154754 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 373662154754' 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: waiting osd.3 seq 373662154754 2026-02-17T05:51:36.615 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:51:36.616 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154755 -lt 373662154754 2026-02-17T05:51:36.616 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:51:36.616 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:36.616 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:36.616 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:51:36.616 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:51:36.626 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/cstats: ceph osd last-stat-seq 1 2026-02-17T05:51:36.626 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 377957122050 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 377957122050 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122051 -lt 377957122050 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-313532612620 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:51:36.627 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-313532612620 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=313532612620 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 313532612620' 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: waiting osd.2 seq 313532612620 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 313532612621 -lt 313532612620 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-373662154755 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-373662154755 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:51:36.628 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154755 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 373662154755' 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: waiting osd.3 seq 373662154755 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154755 -lt 373662154755 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:36.629 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:51:36.942 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: eph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:36.942 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:36.942 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:36.942 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:36.942 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:36.942 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:166928: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:166928: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:51:36.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:51:36.962 INFO:tasks.workunit.client.0.trial074.stderr:wait_for_clean: true 2026-02-17T05:51:36.962 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:51:36.962 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:51:36.962 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:51:36.962 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:51:36.962 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:166925: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:51:36.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 166924 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:51:36.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:51:36.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:51:36.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:51:36.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:51:36.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:51:36.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:51:36.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:36.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:36.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:36.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:36.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:37.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:37.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:51:37.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:51:37.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:51:37.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:51:37.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:51:37.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:51:37.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:51:37.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:51:37.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:51:37.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:51:37.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:51:37.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:51:37.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:51:37.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:51:37.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:51:37.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:51:37.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:51:37.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:51:37.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:51:37.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:51:37.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:51:37.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:51:37.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:51:37.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:51:37.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:51:37.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:51:37.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:51:37.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:51:37.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:51:37.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:51:37.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:51:37.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:51:37.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:51:37.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:51:37.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:51:37.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:51:37.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:51:37.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:51:37.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:51:37.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:51:37.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:51:37.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:51:37.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:51:37.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:51:37.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:51:37.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:51:37.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:51:37.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:51:37.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:51:37.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:51:37.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:51:37.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:51:37.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:51:37.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:51:37.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_lrc_appends td/osd-scrub-repair 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:842: TEST_corrupt_and_repair_lrc_appends: corrupt_and_repair_lrc td/osd-scrub-repair false 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:825: corrupt_and_repair_lrc: local dir=td/osd-scrub-repair 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:826: corrupt_and_repair_lrc: local allow_overwrites=false 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:827: corrupt_and_repair_lrc: local poolname=ecpool 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:829: corrupt_and_repair_lrc: run_mon td/osd-scrub-repair a 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:51:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:51:37.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:51:37.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:37.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:37.169 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:37.169 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.170 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:37.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:51:37.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:51:37.201 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:51:37.202 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:51:37.202 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.202 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.202 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:51:37.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:51:37.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:51:37.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:51:37.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:51:37.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:51:37.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:51:37.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:51:37.258 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:51:37.259 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:51:37.259 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:51:37.259 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:51:37.259 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.259 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.259 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:51:37.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:51:37.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:51:37.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:830: corrupt_and_repair_lrc: run_mgr td/osd-scrub-repair x 2026-02-17T05:51:37.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:51:37.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:51:37.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:51:37.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:51:37.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:51:37.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:51:37.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:51:37.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:37.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:37.407 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:37.407 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.407 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:37.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:51:37.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:51:37.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: seq 0 9 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 0 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:37.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:51:37.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:51:37.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:51:37.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b97b3517-4b01-4983-8d3a-db30e8643925 2026-02-17T05:51:37.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b97b3517-4b01-4983-8d3a-db30e8643925' 2026-02-17T05:51:37.431 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 b97b3517-4b01-4983-8d3a-db30e8643925 2026-02-17T05:51:37.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:51:37.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDpAZRp+JKsGRAAHEINekByVfHFqweSi9mh5w== 2026-02-17T05:51:37.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDpAZRp+JKsGRAAHEINekByVfHFqweSi9mh5w=="}' 2026-02-17T05:51:37.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b97b3517-4b01-4983-8d3a-db30e8643925 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:51:37.533 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:51:37.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:51:37.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDpAZRp+JKsGRAAHEINekByVfHFqweSi9mh5w== --osd-uuid b97b3517-4b01-4983-8d3a-db30e8643925 2026-02-17T05:51:37.557 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:37.543+0000 7fb35fc2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:37.559 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:37.543+0000 7fb35fc2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:37.560 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:37.547+0000 7fb35fc2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:37.560 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:37.547+0000 7fb35fc2f8c0 -1 bdev(0x55690dd51c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:51:37.560 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:37.547+0000 7fb35fc2f8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:51:40.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:51:40.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:51:40.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:51:40.370 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:51:40.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:51:40.560 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:51:40.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:51:40.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:51:40.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:51:40.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:51:40.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:51:40.584 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:40.571+0000 7fb00817e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:40.585 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:40.571+0000 7fb00817e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:40.586 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:40.571+0000 7fb00817e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:40.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:51:40.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:40.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:51:40.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:40.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:40.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:40.718 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:51:40.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:40.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:40.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:51:40.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:41.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:41.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:41.869 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:51:41.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:41.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:41.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:51:42.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:42.089 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:42.075+0000 7fb00817e8c0 -1 Falling back to public interface 2026-02-17T05:51:43.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:43.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:43.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:43.018 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:51:43.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:43.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:51:43.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:44.168 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:51:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:51:44.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:45.080 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:45.067+0000 7fb00817e8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:51:45.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:45.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:45.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:45.319 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:51:45.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:45.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:51:45.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:46.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:46.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:46.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:51:46.471 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:51:46.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:46.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2671546151,v1:127.0.0.1:6803/2671546151] [v2:127.0.0.1:6804/2671546151,v1:127.0.0.1:6805/2671546151] exists,up b97b3517-4b01-4983-8d3a-db30e8643925 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 1 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:51:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:46.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:51:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:51:46.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:51:46.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9622b131-8346-4893-b78c-b6c0aa125c64 2026-02-17T05:51:46.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 9622b131-8346-4893-b78c-b6c0aa125c64' 2026-02-17T05:51:46.625 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 9622b131-8346-4893-b78c-b6c0aa125c64 2026-02-17T05:51:46.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:51:46.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDyAZRpHv5UJRAAVCKmSbuxAB3pYwnbuDr/8w== 2026-02-17T05:51:46.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDyAZRpHv5UJRAAVCKmSbuxAB3pYwnbuDr/8w=="}' 2026-02-17T05:51:46.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9622b131-8346-4893-b78c-b6c0aa125c64 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:51:46.782 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:51:46.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:51:46.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDyAZRpHv5UJRAAVCKmSbuxAB3pYwnbuDr/8w== --osd-uuid 9622b131-8346-4893-b78c-b6c0aa125c64 2026-02-17T05:51:46.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:46.795+0000 7f39955928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:46.809 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:46.795+0000 7f39955928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:46.810 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:46.795+0000 7f39955928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:46.811 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:46.795+0000 7f39955928c0 -1 bdev(0x556c6b9e7c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:51:46.811 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:46.795+0000 7f39955928c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:51:49.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:51:49.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:51:49.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:51:49.102 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:51:49.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:51:49.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:51:49.300 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:51:49.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:51:49.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:51:49.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:51:49.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:51:49.316 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:49.303+0000 7f9de1ef98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:49.317 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:49.303+0000 7f9de1ef98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:49.318 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:49.303+0000 7f9de1ef98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:51:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:51:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:49.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:49.452 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:51:49.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:49.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:49.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:49.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:50.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:50.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:50.604 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:51:50.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:50.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:50.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:50.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:50.853 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:50.839+0000 7f9de1ef98c0 -1 Falling back to public interface 2026-02-17T05:51:51.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:51.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:51.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:51:51.753 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:51:51.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:51.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:51.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:52.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:52.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:52.902 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:51:52.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:51:52.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:52.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:53.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:53.876 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:53.863+0000 7f9de1ef98c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:51:54.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:54.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:54.053 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:51:54.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:51:54.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:54.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:54.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:55.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:55.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:55.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:51:55.210 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:51:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:51:55.361 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/82625870,v1:127.0.0.1:6811/82625870] [v2:127.0.0.1:6812/82625870,v1:127.0.0.1:6813/82625870] exists,up 9622b131-8346-4893-b78c-b6c0aa125c64 2026-02-17T05:51:55.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 2 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:51:55.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:51:55.366 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:51:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:51:55.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:51:55.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:51:55.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:51:55.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:51:55.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:51:55.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=269465dd-ab58-4b44-a888-c9b416d76578 2026-02-17T05:51:55.369 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 269465dd-ab58-4b44-a888-c9b416d76578 2026-02-17T05:51:55.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 269465dd-ab58-4b44-a888-c9b416d76578' 2026-02-17T05:51:55.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:51:55.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD7AZRpL07fFRAAzWnrXBrdLK32CHkzSxTwDg== 2026-02-17T05:51:55.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD7AZRpL07fFRAAzWnrXBrdLK32CHkzSxTwDg=="}' 2026-02-17T05:51:55.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 269465dd-ab58-4b44-a888-c9b416d76578 -i td/osd-scrub-repair/2/new.json 2026-02-17T05:51:55.531 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:51:55.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:51:55.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD7AZRpL07fFRAAzWnrXBrdLK32CHkzSxTwDg== --osd-uuid 269465dd-ab58-4b44-a888-c9b416d76578 2026-02-17T05:51:55.556 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:55.543+0000 7fde83f668c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:55.557 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:55.543+0000 7fde83f668c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:55.558 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:55.543+0000 7fde83f668c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:55.559 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:55.543+0000 7fde83f668c0 -1 bdev(0x55fced331c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:51:55.559 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:55.543+0000 7fde83f668c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:51:57.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:51:57.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:51:57.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:51:57.843 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:51:57.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:51:58.036 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:51:58.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:51:58.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:51:58.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:51:58.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:51:58.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:51:58.053 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:58.039+0000 7f3f851ec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:58.054 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:58.039+0000 7f3f851ec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:58.055 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:58.039+0000 7f3f851ec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:51:58.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:51:58.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:51:58.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:51:58.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:51:58.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:51:58.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:58.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:51:58.193 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:51:58.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:58.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:51:58.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:59.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:51:59.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:51:59.347 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:51:59.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:51:59.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:51:59.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:51:59.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:51:59.553 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:51:59.539+0000 7f3f851ec8c0 -1 Falling back to public interface 2026-02-17T05:52:00.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:00.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:00.499 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:52:00.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:52:00.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:00.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:52:00.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:01.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:01.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:01.652 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:01.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:52:01.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:01.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:52:01.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:02.563 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:02.547+0000 7f3f851ec8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:52:02.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:02.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:02.802 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:02.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:52:02.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:02.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:52:02.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:03.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:03.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:03.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:52:03.955 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:03.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:03.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:52:04.104 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2657843849,v1:127.0.0.1:6819/2657843849] [v2:127.0.0.1:6820/2657843849,v1:127.0.0.1:6821/2657843849] exists,up 269465dd-ab58-4b44-a888-c9b416d76578 2026-02-17T05:52:04.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:52:04.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:52:04.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:52:04.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 3 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:52:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:52:04.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:52:04.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:52:04.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:52:04.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:52:04.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5f5b00fb-b073-443d-8e8a-30731fc0db9c 2026-02-17T05:52:04.112 INFO:tasks.workunit.client.0.trial074.stdout:add osd3 5f5b00fb-b073-443d-8e8a-30731fc0db9c 2026-02-17T05:52:04.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 5f5b00fb-b073-443d-8e8a-30731fc0db9c' 2026-02-17T05:52:04.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:52:04.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAEApRp9PGQBhAAjnAML7fQsGKuh0McuBpQKw== 2026-02-17T05:52:04.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAEApRp9PGQBhAAjnAML7fQsGKuh0McuBpQKw=="}' 2026-02-17T05:52:04.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5f5b00fb-b073-443d-8e8a-30731fc0db9c -i td/osd-scrub-repair/3/new.json 2026-02-17T05:52:04.274 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:04.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T05:52:04.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAEApRp9PGQBhAAjnAML7fQsGKuh0McuBpQKw== --osd-uuid 5f5b00fb-b073-443d-8e8a-30731fc0db9c 2026-02-17T05:52:04.300 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:04.287+0000 7f2a763978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:04.302 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:04.287+0000 7f2a763978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:04.303 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:04.287+0000 7f2a763978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:04.303 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:04.287+0000 7f2a763978c0 -1 bdev(0x55c61bfdbc00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T05:52:04.303 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:04.287+0000 7f2a763978c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T05:52:06.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T05:52:06.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:52:06.813 INFO:tasks.workunit.client.0.trial074.stdout:adding osd3 key to auth repository 2026-02-17T05:52:06.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T05:52:06.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:52:07.004 INFO:tasks.workunit.client.0.trial074.stdout:start osd.3 2026-02-17T05:52:07.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T05:52:07.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:52:07.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:52:07.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:52:07.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:52:07.028 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:07.015+0000 7f2bd1e258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:07.030 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:07.015+0000 7f2bd1e258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:07.031 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:07.015+0000 7f2bd1e258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:07.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T05:52:07.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:52:07.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:52:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:52:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:52:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:52:07.158 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:52:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:52:07.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:08.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:08.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:08.310 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:52:08.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:52:08.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:08.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:52:08.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:08.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:08.523+0000 7f2bd1e258c0 -1 Falling back to public interface 2026-02-17T05:52:09.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:09.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:09.462 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:52:09.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:52:09.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:09.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:52:09.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:10.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:10.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:10.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:52:10.610 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:10.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:10.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:52:10.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:11.512 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:11.499+0000 7f2bd1e258c0 -1 osd.3 0 log_to_monitors true 2026-02-17T05:52:11.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:52:11.759 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:52:11.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:12.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:12.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:12.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:52:12.909 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:12.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:12.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:52:13.056 INFO:tasks.workunit.client.0.trial074.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3000802902,v1:127.0.0.1:6827/3000802902] [v2:127.0.0.1:6828/3000802902,v1:127.0.0.1:6829/3000802902] exists,up 5f5b00fb-b073-443d-8e8a-30731fc0db9c 2026-02-17T05:52:13.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 4 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/4 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/4' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/4/journal' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:52:13.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:52:13.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/4 2026-02-17T05:52:13.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:52:13.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5d6cf6f1-a314-4a3e-afb1-3f61674bc1fe 2026-02-17T05:52:13.060 INFO:tasks.workunit.client.0.trial074.stdout:add osd4 5d6cf6f1-a314-4a3e-afb1-3f61674bc1fe 2026-02-17T05:52:13.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 5d6cf6f1-a314-4a3e-afb1-3f61674bc1fe' 2026-02-17T05:52:13.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:52:13.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQANApRpwjC3AxAAXjLdzrtxJeYcKPL5G7R4HQ== 2026-02-17T05:52:13.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQANApRpwjC3AxAAXjLdzrtxJeYcKPL5G7R4HQ=="}' 2026-02-17T05:52:13.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5d6cf6f1-a314-4a3e-afb1-3f61674bc1fe -i td/osd-scrub-repair/4/new.json 2026-02-17T05:52:13.217 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:13.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/4/new.json 2026-02-17T05:52:13.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQANApRpwjC3AxAAXjLdzrtxJeYcKPL5G7R4HQ== --osd-uuid 5d6cf6f1-a314-4a3e-afb1-3f61674bc1fe 2026-02-17T05:52:13.241 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:13.227+0000 7f078d76f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:13.242 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:13.227+0000 7f078d76f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:13.243 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:13.231+0000 7f078d76f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:13.244 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:13.231+0000 7f078d76f8c0 -1 bdev(0x55a4d8c55c00 td/osd-scrub-repair/4/block) open stat got: (1) Operation not permitted 2026-02-17T05:52:13.244 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:13.231+0000 7f078d76f8c0 -1 bluestore(td/osd-scrub-repair/4) _read_fsid unparsable uuid 2026-02-17T05:52:15.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/4/keyring 2026-02-17T05:52:15.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:52:15.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T05:52:15.553 INFO:tasks.workunit.client.0.trial074.stdout:adding osd4 key to auth repository 2026-02-17T05:52:15.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:52:15.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T05:52:15.745 INFO:tasks.workunit.client.0.trial074.stdout:start osd.4 2026-02-17T05:52:15.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:52:15.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:52:15.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:52:15.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:52:15.767 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:15.751+0000 7f03fe0bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:15.769 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:15.755+0000 7f03fe0bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:15.770 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:15.755+0000 7f03fe0bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:15.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T05:52:15.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:52:15.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T05:52:15.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:52:15.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:52:15.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:15.899 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:52:15.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:52:15.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:15.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:52:16.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:17.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:17.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:17.050 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:52:17.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:52:17.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:17.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:52:17.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:17.481 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:17.467+0000 7f03fe0bb8c0 -1 Falling back to public interface 2026-02-17T05:52:18.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:18.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:18.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:52:18.201 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:52:18.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:18.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:52:18.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:19.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:19.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:19.353 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:19.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:52:19.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:19.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:52:19.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:52:20.505 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:20.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:52:20.505 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:20.491+0000 7f03fe0bb8c0 -1 osd.4 0 log_to_monitors true 2026-02-17T05:52:20.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:21.658 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:52:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:52:21.808 INFO:tasks.workunit.client.0.trial074.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2400931757,v1:127.0.0.1:6835/2400931757] [v2:127.0.0.1:6836/2400931757,v1:127.0.0.1:6837/2400931757] exists,up 5d6cf6f1-a314-4a3e-afb1-3f61674bc1fe 2026-02-17T05:52:21.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:52:21.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:52:21.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:52:21.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:52:21.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 5 2026-02-17T05:52:21.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:52:21.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:52:21.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:52:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:52:21.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:52:21.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:52:21.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:52:21.817 INFO:tasks.workunit.client.0.trial074.stdout:add osd5 a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:52:21.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 a9207a4f-13df-4249-b5c5-d4f465739a8a' 2026-02-17T05:52:21.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:52:21.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAVApRp0eKBMBAADDjQlEa0/KxSnjV1YODU6g== 2026-02-17T05:52:21.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAVApRp0eKBMBAADDjQlEa0/KxSnjV1YODU6g=="}' 2026-02-17T05:52:21.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a9207a4f-13df-4249-b5c5-d4f465739a8a -i td/osd-scrub-repair/5/new.json 2026-02-17T05:52:21.978 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:21.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/5/new.json 2026-02-17T05:52:21.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAVApRp0eKBMBAADDjQlEa0/KxSnjV1YODU6g== --osd-uuid a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:52:22.003 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:21.991+0000 7f4810fe18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:22.005 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:21.991+0000 7f4810fe18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:22.006 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:21.991+0000 7f4810fe18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:22.006 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:21.991+0000 7f4810fe18c0 -1 bdev(0x5597f2d31c00 td/osd-scrub-repair/5/block) open stat got: (1) Operation not permitted 2026-02-17T05:52:22.007 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:21.991+0000 7f4810fe18c0 -1 bluestore(td/osd-scrub-repair/5) _read_fsid unparsable uuid 2026-02-17T05:52:24.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/5/keyring 2026-02-17T05:52:24.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:52:24.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T05:52:24.309 INFO:tasks.workunit.client.0.trial074.stdout:adding osd5 key to auth repository 2026-02-17T05:52:24.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:52:24.501 INFO:tasks.workunit.client.0.trial074.stdout:start osd.5 2026-02-17T05:52:24.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T05:52:24.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:52:24.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:52:24.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:52:24.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:52:24.524 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:24.511+0000 7fd346c068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:24.526 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:24.511+0000 7fd346c068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:24.527 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:24.515+0000 7fd346c068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:24.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T05:52:24.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:52:24.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:52:24.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:52:24.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:52:24.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:24.653 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:52:24.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:52:24.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:24.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:52:24.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:25.804 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:52:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:52:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:52:25.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:26.009 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:25.995+0000 7fd346c068c0 -1 Falling back to public interface 2026-02-17T05:52:26.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:26.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:26.953 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:52:26.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:52:26.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:26.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:52:27.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:28.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:28.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:28.101 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:28.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:52:28.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:28.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:52:28.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:29.069 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:29.055+0000 7fd346c068c0 -1 osd.5 0 log_to_monitors true 2026-02-17T05:52:29.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:29.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:29.249 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:29.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:52:29.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:29.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:52:29.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:30.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:30.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:30.397 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:30.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:52:30.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:30.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:52:30.552 INFO:tasks.workunit.client.0.trial074.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1774273080,v1:127.0.0.1:6843/1774273080] [v2:127.0.0.1:6844/1774273080,v1:127.0.0.1:6845/1774273080] exists,up a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 6 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/6 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/6' 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/6/journal' 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:52:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:30.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:52:30.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/6 2026-02-17T05:52:30.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:52:30.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7f9630e4-2c40-415b-a7c8-172987404247 2026-02-17T05:52:30.557 INFO:tasks.workunit.client.0.trial074.stdout:add osd6 7f9630e4-2c40-415b-a7c8-172987404247 2026-02-17T05:52:30.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 7f9630e4-2c40-415b-a7c8-172987404247' 2026-02-17T05:52:30.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:52:30.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAeApRpJ3VFIRAADcotiQJNA5GFYgTvAv6Wmw== 2026-02-17T05:52:30.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAeApRpJ3VFIRAADcotiQJNA5GFYgTvAv6Wmw=="}' 2026-02-17T05:52:30.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7f9630e4-2c40-415b-a7c8-172987404247 -i td/osd-scrub-repair/6/new.json 2026-02-17T05:52:30.721 INFO:tasks.workunit.client.0.trial074.stdout:6 2026-02-17T05:52:30.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/6/new.json 2026-02-17T05:52:30.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAeApRpJ3VFIRAADcotiQJNA5GFYgTvAv6Wmw== --osd-uuid 7f9630e4-2c40-415b-a7c8-172987404247 2026-02-17T05:52:30.747 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:30.731+0000 7f8c109368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:30.748 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:30.735+0000 7f8c109368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:30.749 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:30.735+0000 7f8c109368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:30.750 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:30.735+0000 7f8c109368c0 -1 bdev(0x55e9e2741c00 td/osd-scrub-repair/6/block) open stat got: (1) Operation not permitted 2026-02-17T05:52:30.750 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:30.735+0000 7f8c109368c0 -1 bluestore(td/osd-scrub-repair/6) _read_fsid unparsable uuid 2026-02-17T05:52:33.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/6/keyring 2026-02-17T05:52:33.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:52:33.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T05:52:33.273 INFO:tasks.workunit.client.0.trial074.stdout:adding osd6 key to auth repository 2026-02-17T05:52:33.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:52:33.463 INFO:tasks.workunit.client.0.trial074.stdout:start osd.6 2026-02-17T05:52:33.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T05:52:33.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:52:33.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:52:33.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:52:33.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:52:33.486 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:33.471+0000 7f5821d408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:33.488 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:33.475+0000 7f5821d408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:33.489 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:33.475+0000 7f5821d408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:33.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T05:52:33.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:52:33.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T05:52:33.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:52:33.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:52:33.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:33.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:52:33.617 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:52:33.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:33.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:52:33.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:34.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:34.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:34.770 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:52:34.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:52:34.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:34.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:52:34.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:34.993 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:34.979+0000 7f5821d408c0 -1 Falling back to public interface 2026-02-17T05:52:35.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:35.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:35.918 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:52:35.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:52:35.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:35.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:52:36.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:37.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:37.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:37.067 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:52:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:37.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:52:37.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:38.005 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:37.991+0000 7f5821d408c0 -1 osd.6 0 log_to_monitors true 2026-02-17T05:52:38.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:38.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:38.220 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:38.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:52:38.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:38.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:52:38.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:39.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:39.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:39.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:52:39.379 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:39.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:39.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:52:39.529 INFO:tasks.workunit.client.0.trial074.stdout:osd.6 up in weight 1 up_from 34 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1679314051,v1:127.0.0.1:6851/1679314051] [v2:127.0.0.1:6852/1679314051,v1:127.0.0.1:6853/1679314051] exists,up 7f9630e4-2c40-415b-a7c8-172987404247 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 7 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=7 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:52:39.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/7 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/7' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/7/journal' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:52:39.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:52:39.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:52:39.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:52:39.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:52:39.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:52:39.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:52:39.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/7 2026-02-17T05:52:39.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:52:39.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3a435274-353b-4c02-88c3-f08fbba3d719 2026-02-17T05:52:39.537 INFO:tasks.workunit.client.0.trial074.stdout:add osd7 3a435274-353b-4c02-88c3-f08fbba3d719 2026-02-17T05:52:39.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd7 3a435274-353b-4c02-88c3-f08fbba3d719' 2026-02-17T05:52:39.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:52:39.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAnApRplt34HxAAj8BJF8sc0AAfG8ozy+DWfw== 2026-02-17T05:52:39.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAnApRplt34HxAAj8BJF8sc0AAfG8ozy+DWfw=="}' 2026-02-17T05:52:39.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3a435274-353b-4c02-88c3-f08fbba3d719 -i td/osd-scrub-repair/7/new.json 2026-02-17T05:52:39.698 INFO:tasks.workunit.client.0.trial074.stdout:7 2026-02-17T05:52:39.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/7/new.json 2026-02-17T05:52:39.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 7 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAnApRplt34HxAAj8BJF8sc0AAfG8ozy+DWfw== --osd-uuid 3a435274-353b-4c02-88c3-f08fbba3d719 2026-02-17T05:52:39.725 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:39.711+0000 7fa3c3c148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:39.726 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:39.711+0000 7fa3c3c148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:39.727 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:39.711+0000 7fa3c3c148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:39.728 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:39.715+0000 7fa3c3c148c0 -1 bdev(0x559a50969c00 td/osd-scrub-repair/7/block) open stat got: (1) Operation not permitted 2026-02-17T05:52:39.728 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:39.715+0000 7fa3c3c148c0 -1 bluestore(td/osd-scrub-repair/7) _read_fsid unparsable uuid 2026-02-17T05:52:42.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/7/keyring 2026-02-17T05:52:42.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:52:42.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd7 key to auth repository 2026-02-17T05:52:42.002 INFO:tasks.workunit.client.0.trial074.stdout:adding osd7 key to auth repository 2026-02-17T05:52:42.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:52:42.195 INFO:tasks.workunit.client.0.trial074.stdout:start osd.7 2026-02-17T05:52:42.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.7 2026-02-17T05:52:42.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 7 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:52:42.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:52:42.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:52:42.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:52:42.220 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:42.207+0000 7ff43844b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:42.221 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:42.207+0000 7ff43844b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:42.223 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:42.207+0000 7ff43844b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:42.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 7 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=7 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:42.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:52:42.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:43.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:43.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:43.504 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:52:43.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:52:43.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:43.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:52:43.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:43.929 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:43.915+0000 7ff43844b8c0 -1 Falling back to public interface 2026-02-17T05:52:44.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:44.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:44.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:52:44.652 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:52:44.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:44.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:52:44.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:45.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:45.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:45.806 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:45.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:52:45.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:45.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:52:45.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:46.906 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:46.891+0000 7ff43844b8c0 -1 osd.7 0 log_to_monitors true 2026-02-17T05:52:46.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:46.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:46.955 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:46.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:52:46.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:46.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:52:47.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:48.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:48.102 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:48.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:48.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:52:48.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:48.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:52:48.248 INFO:tasks.workunit.client.0.trial074.stdout:osd.7 up in weight 1 up_from 38 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/637077721,v1:127.0.0.1:6859/637077721] [v2:127.0.0.1:6860/637077721,v1:127.0.0.1:6861/637077721] exists,up 3a435274-353b-4c02-88c3-f08fbba3d719 2026-02-17T05:52:48.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:52:48.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:52:48.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:52:48.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:52:48.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 8 2026-02-17T05:52:48.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:52:48.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=8 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/8 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/8' 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/8/journal' 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:52:48.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:52:48.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/8 2026-02-17T05:52:48.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:52:48.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6aad6721-79f0-4a12-bf3e-38bab6b72ffc 2026-02-17T05:52:48.256 INFO:tasks.workunit.client.0.trial074.stdout:add osd8 6aad6721-79f0-4a12-bf3e-38bab6b72ffc 2026-02-17T05:52:48.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd8 6aad6721-79f0-4a12-bf3e-38bab6b72ffc' 2026-02-17T05:52:48.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:52:48.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAwApRpErItDxAArCNkX+kLa3HPBK5PCwyB+A== 2026-02-17T05:52:48.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAwApRpErItDxAArCNkX+kLa3HPBK5PCwyB+A=="}' 2026-02-17T05:52:48.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6aad6721-79f0-4a12-bf3e-38bab6b72ffc -i td/osd-scrub-repair/8/new.json 2026-02-17T05:52:48.419 INFO:tasks.workunit.client.0.trial074.stdout:8 2026-02-17T05:52:48.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/8/new.json 2026-02-17T05:52:48.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 8 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAwApRpErItDxAArCNkX+kLa3HPBK5PCwyB+A== --osd-uuid 6aad6721-79f0-4a12-bf3e-38bab6b72ffc 2026-02-17T05:52:48.444 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:48.431+0000 7f02e4d4e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:48.446 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:48.431+0000 7f02e4d4e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:48.447 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:48.431+0000 7f02e4d4e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:48.447 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:48.435+0000 7f02e4d4e8c0 -1 bdev(0x55b38d023c00 td/osd-scrub-repair/8/block) open stat got: (1) Operation not permitted 2026-02-17T05:52:48.448 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:48.435+0000 7f02e4d4e8c0 -1 bluestore(td/osd-scrub-repair/8) _read_fsid unparsable uuid 2026-02-17T05:52:50.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/8/keyring 2026-02-17T05:52:50.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:52:50.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd8 key to auth repository 2026-02-17T05:52:50.747 INFO:tasks.workunit.client.0.trial074.stdout:adding osd8 key to auth repository 2026-02-17T05:52:50.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:52:50.932 INFO:tasks.workunit.client.0.trial074.stdout:start osd.8 2026-02-17T05:52:50.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.8 2026-02-17T05:52:50.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 8 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:52:50.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:52:50.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:52:50.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:52:50.954 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:50.939+0000 7f5da02e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:50.956 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:50.943+0000 7f5da02e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:50.957 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:50.943+0000 7f5da02e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:51.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 8 2026-02-17T05:52:51.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:52:51.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=8 2026-02-17T05:52:51.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:52:51.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:52:51.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:51.085 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:52:51.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:52:51.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:51.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:52:51.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:52.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:52.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:52.236 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:52:52.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:52:52.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:52.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:52:52.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:52.485 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:52.471+0000 7f5da02e08c0 -1 Falling back to public interface 2026-02-17T05:52:53.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:53.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:53.387 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:52:53.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:52:53.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:53.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:52:53.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:54.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:54.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:54.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:52:54.543 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:52:54.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:54.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:52:54.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:55.475 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:55.459+0000 7f5da02e08c0 -1 osd.8 0 log_to_monitors true 2026-02-17T05:52:55.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:55.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:55.699 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:52:55.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:52:55.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:52:55.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:55.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:52:56.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:52:56.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:56.848 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:52:56.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:52:56.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:56.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:52:56.996 INFO:tasks.workunit.client.0.trial074.stdout:osd.8 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/2223318652,v1:127.0.0.1:6867/2223318652] [v2:127.0.0.1:6868/2223318652,v1:127.0.0.1:6869/2223318652] exists,up 6aad6721-79f0-4a12-bf3e-38bab6b72ffc 2026-02-17T05:52:57.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:52:57.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:52:57.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 9 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=9 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:52:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:52:57.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:52:57.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:52:57.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:52:57.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:52:57.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:52:57.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:52:57.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:52:57.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:52:57.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:52:57.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:52:57.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:52:57.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:52:57.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:52:57.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:52:57.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:57.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:52:57.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:52:57.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:52:57.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:52:57.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:52:57.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:52:57.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:52:57.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:52:57.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:52:57.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:52:57.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:52:57.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:52:57.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:52:57.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:52:57.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:52:57.008 INFO:tasks.workunit.client.0.trial074.stdout:add osd9 eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:52:57.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd9 eb323775-696d-4501-887c-ef9fa5171773' 2026-02-17T05:52:57.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:52:57.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA5ApRpY3ocABAAAvkOXZ6WGTpubYdigGHg+g== 2026-02-17T05:52:57.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA5ApRpY3ocABAAAvkOXZ6WGTpubYdigGHg+g=="}' 2026-02-17T05:52:57.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new eb323775-696d-4501-887c-ef9fa5171773 -i td/osd-scrub-repair/9/new.json 2026-02-17T05:52:57.167 INFO:tasks.workunit.client.0.trial074.stdout:9 2026-02-17T05:52:57.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/9/new.json 2026-02-17T05:52:57.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5ApRpY3ocABAAAvkOXZ6WGTpubYdigGHg+g== --osd-uuid eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:52:57.195 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:57.179+0000 7f7565e1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:57.197 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:57.183+0000 7f7565e1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:57.198 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:57.183+0000 7f7565e1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:57.198 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:57.183+0000 7f7565e1b8c0 -1 bdev(0x563063047c00 td/osd-scrub-repair/9/block) open stat got: (1) Operation not permitted 2026-02-17T05:52:57.199 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:57.183+0000 7f7565e1b8c0 -1 bluestore(td/osd-scrub-repair/9) _read_fsid unparsable uuid 2026-02-17T05:52:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/9/keyring 2026-02-17T05:52:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:52:59.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd9 key to auth repository 2026-02-17T05:52:59.502 INFO:tasks.workunit.client.0.trial074.stdout:adding osd9 key to auth repository 2026-02-17T05:52:59.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:52:59.695 INFO:tasks.workunit.client.0.trial074.stdout:start osd.9 2026-02-17T05:52:59.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.9 2026-02-17T05:52:59.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:52:59.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:52:59.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:52:59.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:52:59.719 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:59.703+0000 7fc365e408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:59.721 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:59.707+0000 7fc365e408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:59.722 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:52:59.707+0000 7fc365e408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:52:59.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 9 2026-02-17T05:52:59.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:52:59.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:52:59.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:52:59.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:52:59.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:52:59.846 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:52:59.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:52:59.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:52:59.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:52:59.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:00.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:00.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:00.998 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:53:00.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:53:00.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:00.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:01.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:01.225 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:01.211+0000 7fc365e408c0 -1 Falling back to public interface 2026-02-17T05:53:02.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:02.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:02.153 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:53:02.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:53:02.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:02.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:02.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:03.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:03.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:03.301 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:53:03.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:53:03.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:03.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:03.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:04.219 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:04.203+0000 7fc365e408c0 -1 osd.9 0 log_to_monitors true 2026-02-17T05:53:04.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:04.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:04.450 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:53:04.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:53:04.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:04.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:04.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:05.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:05.615 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:53:05.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:05.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:53:05.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:05.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:05.760 INFO:tasks.workunit.client.0.trial074.stdout:osd.9 up in weight 1 up_from 46 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/4125267140,v1:127.0.0.1:6875/4125267140] [v2:127.0.0.1:6876/4125267140,v1:127.0.0.1:6877/4125267140] exists,up eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:53:05.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:53:05.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:53:05.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:53:05.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:834: corrupt_and_repair_lrc: create_rbd_pool 2026-02-17T05:53:05.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:53:05.900 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:53:05.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:53:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:53:06.121 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:53:06.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:53:07.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:53:07.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:835: corrupt_and_repair_lrc: wait_for_clean 2026-02-17T05:53:07.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:53:07.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:53:07.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:53:07.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:53:07.371 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:53:07.371 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:53:07.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:53:07.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:53:07.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:53:07.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:53:07.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:53:07.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:53:07.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:53:07.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:53:07.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:53:07.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:53:07.638 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:07.638 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:07.638 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:07.638 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:07.638 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:53:07.639 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:53:07.639 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:53:07.639 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:53:07.639 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:53:07.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:53:07.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:07.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:53:07.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836499 2026-02-17T05:53:07.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836499 2026-02-17T05:53:07.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499' 2026-02-17T05:53:07.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:07.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:53:07.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672976 2026-02-17T05:53:07.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672976 2026-02-17T05:53:07.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976' 2026-02-17T05:53:07.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:07.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:53:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509454 2026-02-17T05:53:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509454 2026-02-17T05:53:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454' 2026-02-17T05:53:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:07.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:53:07.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345933 2026-02-17T05:53:07.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345933 2026-02-17T05:53:07.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454 3-85899345933' 2026-02-17T05:53:07.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:07.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:53:07.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182411 2026-02-17T05:53:07.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182411 2026-02-17T05:53:07.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454 3-85899345933 4-107374182411' 2026-02-17T05:53:07.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:07.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:53:08.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018889 2026-02-17T05:53:08.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018889 2026-02-17T05:53:08.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454 3-85899345933 4-107374182411 5-128849018889' 2026-02-17T05:53:08.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:08.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:53:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888071 2026-02-17T05:53:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888071 2026-02-17T05:53:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454 3-85899345933 4-107374182411 5-128849018889 6-146028888071' 2026-02-17T05:53:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:08.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:53:08.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757254 2026-02-17T05:53:08.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757254 2026-02-17T05:53:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454 3-85899345933 4-107374182411 5-128849018889 6-146028888071 7-163208757254' 2026-02-17T05:53:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:08.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:53:08.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626436 2026-02-17T05:53:08.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626436 2026-02-17T05:53:08.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454 3-85899345933 4-107374182411 5-128849018889 6-146028888071 7-163208757254 8-180388626436' 2026-02-17T05:53:08.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:08.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:53:08.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=197568495618 2026-02-17T05:53:08.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 197568495618 2026-02-17T05:53:08.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-42949672976 2-64424509454 3-85899345933 4-107374182411 5-128849018889 6-146028888071 7-163208757254 8-180388626436 9-197568495618' 2026-02-17T05:53:08.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:08.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836499 2026-02-17T05:53:08.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:08.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:53:08.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836499 2026-02-17T05:53:08.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:08.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836499 2026-02-17T05:53:08.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-17T05:53:08.353 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836499 2026-02-17T05:53:08.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:08.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836499 2026-02-17T05:53:08.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:53:09.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:53:09.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:09.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836499 2026-02-17T05:53:09.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:53:10.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:53:10.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-17T05:53:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:10.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672976 2026-02-17T05:53:10.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:10.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:53:10.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672976 2026-02-17T05:53:10.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:10.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672976 2026-02-17T05:53:10.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672976' 2026-02-17T05:53:10.819 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672976 2026-02-17T05:53:10.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:53:10.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672976 -lt 42949672976 2026-02-17T05:53:10.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:10.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509454 2026-02-17T05:53:10.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:10.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:53:10.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509454 2026-02-17T05:53:10.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:10.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509454 2026-02-17T05:53:10.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2026-02-17T05:53:10.977 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509454 2026-02-17T05:53:10.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:53:11.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509455 -lt 64424509454 2026-02-17T05:53:11.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:11.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-85899345933 2026-02-17T05:53:11.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:11.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:53:11.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-85899345933 2026-02-17T05:53:11.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:11.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345933 2026-02-17T05:53:11.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 85899345933' 2026-02-17T05:53:11.130 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 85899345933 2026-02-17T05:53:11.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:53:11.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345933 -lt 85899345933 2026-02-17T05:53:11.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:11.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182411 2026-02-17T05:53:11.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:11.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:53:11.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182411 2026-02-17T05:53:11.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:11.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182411 2026-02-17T05:53:11.286 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.4 seq 107374182411 2026-02-17T05:53:11.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182411' 2026-02-17T05:53:11.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:53:11.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182411 -lt 107374182411 2026-02-17T05:53:11.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:11.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-128849018889 2026-02-17T05:53:11.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:11.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:53:11.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-128849018889 2026-02-17T05:53:11.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:11.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018889 2026-02-17T05:53:11.448 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.5 seq 128849018889 2026-02-17T05:53:11.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 128849018889' 2026-02-17T05:53:11.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:53:11.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018889 -lt 128849018889 2026-02-17T05:53:11.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:11.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888071 2026-02-17T05:53:11.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:11.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:53:11.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888071 2026-02-17T05:53:11.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:11.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888071 2026-02-17T05:53:11.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888071' 2026-02-17T05:53:11.603 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.6 seq 146028888071 2026-02-17T05:53:11.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:53:11.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888072 -lt 146028888071 2026-02-17T05:53:11.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:11.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757254 2026-02-17T05:53:11.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:11.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:53:11.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757254 2026-02-17T05:53:11.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:11.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757254 2026-02-17T05:53:11.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757254' 2026-02-17T05:53:11.771 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.7 seq 163208757254 2026-02-17T05:53:11.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:53:11.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757254 -lt 163208757254 2026-02-17T05:53:11.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:11.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626436 2026-02-17T05:53:11.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:11.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:53:11.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626436 2026-02-17T05:53:11.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:11.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626436 2026-02-17T05:53:11.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626436' 2026-02-17T05:53:11.924 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.8 seq 180388626436 2026-02-17T05:53:11.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:53:12.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626437 -lt 180388626436 2026-02-17T05:53:12.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:12.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-197568495618 2026-02-17T05:53:12.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:12.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:53:12.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-197568495618 2026-02-17T05:53:12.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:12.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=197568495618 2026-02-17T05:53:12.081 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.9 seq 197568495618 2026-02-17T05:53:12.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 197568495618' 2026-02-17T05:53:12.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:53:12.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 197568495619 -lt 197568495618 2026-02-17T05:53:12.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:53:12.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:12.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:12.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:53:12.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:53:12.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:53:12.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:53:12.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:53:12.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:53:12.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:53:12.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:53:12.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:53:12.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:53:12.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:12.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:837: corrupt_and_repair_lrc: create_ec_pool ecpool false k=4 m=2 l=3 plugin=lrc 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T05:53:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=4 m=2 l=3 plugin=lrc 2026-02-17T05:53:12.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T05:53:12.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T05:53:13.253 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T05:53:13.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:53:14.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T05:53:14.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T05:53:14.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:53:14.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:53:14.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:53:14.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:53:14.265 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:53:14.265 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:53:14.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:53:14.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:53:14.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:53:14.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:53:14.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:53:14.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:53:14.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:53:14.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:53:14.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:53:14.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:53:14.532 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:53:14.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:53:14.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:14.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:53:14.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T05:53:14.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T05:53:14.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T05:53:14.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:14.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:53:14.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672978 2026-02-17T05:53:14.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672978 2026-02-17T05:53:14.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978' 2026-02-17T05:53:14.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:14.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:53:14.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509457 2026-02-17T05:53:14.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509457 2026-02-17T05:53:14.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457' 2026-02-17T05:53:14.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:14.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:53:14.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345935 2026-02-17T05:53:14.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345935 2026-02-17T05:53:14.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457 3-85899345935' 2026-02-17T05:53:14.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:14.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:53:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182413 2026-02-17T05:53:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182413 2026-02-17T05:53:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457 3-85899345935 4-107374182413' 2026-02-17T05:53:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:14.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:53:14.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018891 2026-02-17T05:53:14.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018891 2026-02-17T05:53:14.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457 3-85899345935 4-107374182413 5-128849018891' 2026-02-17T05:53:14.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:14.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:53:15.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888074 2026-02-17T05:53:15.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888074 2026-02-17T05:53:15.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457 3-85899345935 4-107374182413 5-128849018891 6-146028888074' 2026-02-17T05:53:15.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:15.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:53:15.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757256 2026-02-17T05:53:15.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757256 2026-02-17T05:53:15.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457 3-85899345935 4-107374182413 5-128849018891 6-146028888074 7-163208757256' 2026-02-17T05:53:15.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:15.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:53:15.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626438 2026-02-17T05:53:15.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626438 2026-02-17T05:53:15.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457 3-85899345935 4-107374182413 5-128849018891 6-146028888074 7-163208757256 8-180388626438' 2026-02-17T05:53:15.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:15.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:53:15.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=197568495620 2026-02-17T05:53:15.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 197568495620 2026-02-17T05:53:15.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672978 2-64424509457 3-85899345935 4-107374182413 5-128849018891 6-146028888074 7-163208757256 8-180388626438 9-197568495620' 2026-02-17T05:53:15.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:15.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T05:53:15.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:15.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:53:15.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T05:53:15.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:15.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T05:53:15.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T05:53:15.253 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836501 2026-02-17T05:53:15.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:15.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836501 2026-02-17T05:53:15.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:53:16.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:53:16.404 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:16.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-17T05:53:16.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:16.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672978 2026-02-17T05:53:16.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:16.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:53:16.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672978 2026-02-17T05:53:16.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:16.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672978 2026-02-17T05:53:16.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672978' 2026-02-17T05:53:16.560 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672978 2026-02-17T05:53:16.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:53:16.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672979 -lt 42949672978 2026-02-17T05:53:16.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:16.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509457 2026-02-17T05:53:16.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:16.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:53:16.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509457 2026-02-17T05:53:16.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:16.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509457 2026-02-17T05:53:16.718 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509457 2026-02-17T05:53:16.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509457' 2026-02-17T05:53:16.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:53:16.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509457 -lt 64424509457 2026-02-17T05:53:16.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:16.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-85899345935 2026-02-17T05:53:16.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:16.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:53:16.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-85899345935 2026-02-17T05:53:16.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:16.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345935 2026-02-17T05:53:16.886 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 85899345935 2026-02-17T05:53:16.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 85899345935' 2026-02-17T05:53:16.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:53:17.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345935 -lt 85899345935 2026-02-17T05:53:17.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:17.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182413 2026-02-17T05:53:17.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:17.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:53:17.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182413 2026-02-17T05:53:17.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:17.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182413 2026-02-17T05:53:17.052 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.4 seq 107374182413 2026-02-17T05:53:17.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182413' 2026-02-17T05:53:17.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:53:17.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182413 -lt 107374182413 2026-02-17T05:53:17.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:17.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-128849018891 2026-02-17T05:53:17.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:17.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:53:17.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-128849018891 2026-02-17T05:53:17.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:17.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018891 2026-02-17T05:53:17.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 128849018891' 2026-02-17T05:53:17.212 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.5 seq 128849018891 2026-02-17T05:53:17.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:53:17.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018892 -lt 128849018891 2026-02-17T05:53:17.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:17.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888074 2026-02-17T05:53:17.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:17.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:53:17.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888074 2026-02-17T05:53:17.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:17.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888074 2026-02-17T05:53:17.364 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.6 seq 146028888074 2026-02-17T05:53:17.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888074' 2026-02-17T05:53:17.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:53:17.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888074 -lt 146028888074 2026-02-17T05:53:17.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:17.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757256 2026-02-17T05:53:17.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:17.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:53:17.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757256 2026-02-17T05:53:17.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:17.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757256 2026-02-17T05:53:17.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757256' 2026-02-17T05:53:17.520 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.7 seq 163208757256 2026-02-17T05:53:17.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:53:17.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757256 -lt 163208757256 2026-02-17T05:53:17.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:17.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626438 2026-02-17T05:53:17.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:17.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:53:17.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626438 2026-02-17T05:53:17.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:17.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626438 2026-02-17T05:53:17.674 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.8 seq 180388626438 2026-02-17T05:53:17.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626438' 2026-02-17T05:53:17.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:53:17.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626439 -lt 180388626438 2026-02-17T05:53:17.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:17.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-197568495620 2026-02-17T05:53:17.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:17.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:53:17.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-197568495620 2026-02-17T05:53:17.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:17.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=197568495620 2026-02-17T05:53:17.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 197568495620' 2026-02-17T05:53:17.827 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.9 seq 197568495620 2026-02-17T05:53:17.828 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:53:17.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 197568495621 -lt 197568495620 2026-02-17T05:53:17.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:53:17.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:17.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:18.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:53:18.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:53:18.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:53:18.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:53:18.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:53:18.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:53:18.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:53:18.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:53:18.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:53:18.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:53:18.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:18.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:838: corrupt_and_repair_lrc: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T05:53:18.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T05:53:18.687 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T05:53:18.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T05:53:18.891 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T05:53:18.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T05:53:18.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T05:53:18.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:53:18.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T05:53:18.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:53:18.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:53:18.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:53:18.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:53:19.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T05:53:19.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T05:53:19.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T05:53:19.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T05:53:19.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T05:53:19.089 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:53:19.089 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr:9 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr:7' 2026-02-17T05:53:19.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 9 0 6 2 1 7 2026-02-17T05:53:19.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('5' '9' '0' '6' '2' '1' '7') 2026-02-17T05:53:19.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T05:53:19.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=5 2026-02-17T05:53:19.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=9 2026-02-17T05:53:19.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T05:53:19.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:53:19.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:53:19.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:53:19.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:53:19.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:53:19.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:53:19.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:53:19.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:53:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:53:21.203 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:53:21.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:53:21.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:53:21.748 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:53:21.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:53:21.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:53:21.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:53:21.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:53:21.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:53:21.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:53:21.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:53:21.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:53:21.764 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:21.750+0000 7f1cb1c4a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:21.765 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:21.750+0000 7f1cb1c4a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:21.766 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:21.750+0000 7f1cb1c4a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:21.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:21.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:53:22.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:23.017 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:23.002+0000 7f1cb1c4a8c0 -1 Falling back to public interface 2026-02-17T05:53:23.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:53:23.056 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:53:23.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:24.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:24.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:24.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:53:24.205 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:24.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:24.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:53:24.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:25.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:25.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:25.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:53:25.358 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:25.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:25.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:53:25.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:26.036 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:26.022+0000 7f1cb1c4a8c0 -1 osd.3 61 log_to_monitors true 2026-02-17T05:53:26.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:26.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:26.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:53:26.513 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:26.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:26.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:53:26.663 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 65 up_thru 65 down_at 62 last_clean_interval [20,61) [v2:127.0.0.1:6826/213166029,v1:127.0.0.1:6827/213166029] [v2:127.0.0.1:6828/213166029,v1:127.0.0.1:6829/213166029] exists,up 5f5b00fb-b073-443d-8e8a-30731fc0db9c 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:53:26.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:53:26.665 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:53:26.665 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:53:26.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:53:26.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:53:26.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:53:26.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:53:26.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:53:26.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:53:26.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:53:26.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:53:26.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:53:26.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:53:26.929 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:26.929 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:26.929 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:26.929 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:26.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:53:27.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836505 2026-02-17T05:53:27.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836505 2026-02-17T05:53:27.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505' 2026-02-17T05:53:27.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:53:27.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672982 2026-02-17T05:53:27.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672982 2026-02-17T05:53:27.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982' 2026-02-17T05:53:27.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:53:27.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509460 2026-02-17T05:53:27.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509460 2026-02-17T05:53:27.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460' 2026-02-17T05:53:27.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:53:27.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874242 2026-02-17T05:53:27.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874242 2026-02-17T05:53:27.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460 3-279172874242' 2026-02-17T05:53:27.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:53:27.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182417 2026-02-17T05:53:27.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182417 2026-02-17T05:53:27.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460 3-279172874242 4-107374182417' 2026-02-17T05:53:27.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:53:27.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018895 2026-02-17T05:53:27.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018895 2026-02-17T05:53:27.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460 3-279172874242 4-107374182417 5-128849018895' 2026-02-17T05:53:27.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:53:27.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888077 2026-02-17T05:53:27.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888077 2026-02-17T05:53:27.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460 3-279172874242 4-107374182417 5-128849018895 6-146028888077' 2026-02-17T05:53:27.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:53:27.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757259 2026-02-17T05:53:27.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757259 2026-02-17T05:53:27.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460 3-279172874242 4-107374182417 5-128849018895 6-146028888077 7-163208757259' 2026-02-17T05:53:27.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:53:27.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626442 2026-02-17T05:53:27.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626442 2026-02-17T05:53:27.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460 3-279172874242 4-107374182417 5-128849018895 6-146028888077 7-163208757259 8-180388626442' 2026-02-17T05:53:27.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:27.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:53:27.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=197568495624 2026-02-17T05:53:27.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 197568495624 2026-02-17T05:53:27.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836505 1-42949672982 2-64424509460 3-279172874242 4-107374182417 5-128849018895 6-146028888077 7-163208757259 8-180388626442 9-197568495624' 2026-02-17T05:53:27.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:27.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836505 2026-02-17T05:53:27.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:27.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:53:27.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836505 2026-02-17T05:53:27.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:27.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836505 2026-02-17T05:53:27.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-17T05:53:27.658 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836505 2026-02-17T05:53:27.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:27.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836505 -lt 21474836505 2026-02-17T05:53:27.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:27.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672982 2026-02-17T05:53:27.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:27.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:53:27.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672982 2026-02-17T05:53:27.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:27.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672982 2026-02-17T05:53:27.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672982' 2026-02-17T05:53:27.821 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672982 2026-02-17T05:53:27.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:53:27.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672982 -lt 42949672982 2026-02-17T05:53:27.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:27.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509460 2026-02-17T05:53:27.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:27.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:53:27.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509460 2026-02-17T05:53:27.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:27.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509460 2026-02-17T05:53:27.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509460' 2026-02-17T05:53:27.979 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509460 2026-02-17T05:53:27.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:53:28.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509460 -lt 64424509460 2026-02-17T05:53:28.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:28.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874242 2026-02-17T05:53:28.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:28.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:53:28.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874242 2026-02-17T05:53:28.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:28.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874242 2026-02-17T05:53:28.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874242' 2026-02-17T05:53:28.143 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874242 2026-02-17T05:53:28.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:53:28.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874242 -lt 279172874242 2026-02-17T05:53:28.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:28.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182417 2026-02-17T05:53:28.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:28.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:53:28.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182417 2026-02-17T05:53:28.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:28.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182417 2026-02-17T05:53:28.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182417' 2026-02-17T05:53:28.311 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 107374182417 2026-02-17T05:53:28.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:53:28.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182417 -lt 107374182417 2026-02-17T05:53:28.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:28.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-128849018895 2026-02-17T05:53:28.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:28.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:53:28.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-128849018895 2026-02-17T05:53:28.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:28.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018895 2026-02-17T05:53:28.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 128849018895' 2026-02-17T05:53:28.472 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 128849018895 2026-02-17T05:53:28.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:53:28.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018895 -lt 128849018895 2026-02-17T05:53:28.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:28.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888077 2026-02-17T05:53:28.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:28.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:53:28.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888077 2026-02-17T05:53:28.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:28.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888077 2026-02-17T05:53:28.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888077' 2026-02-17T05:53:28.628 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 146028888077 2026-02-17T05:53:28.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:53:28.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888077 -lt 146028888077 2026-02-17T05:53:28.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:28.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757259 2026-02-17T05:53:28.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:28.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:53:28.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757259 2026-02-17T05:53:28.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:28.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757259 2026-02-17T05:53:28.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757259' 2026-02-17T05:53:28.792 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 163208757259 2026-02-17T05:53:28.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:53:28.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757259 -lt 163208757259 2026-02-17T05:53:28.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:28.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626442 2026-02-17T05:53:28.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:28.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:53:28.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626442 2026-02-17T05:53:28.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:28.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626442 2026-02-17T05:53:28.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626442' 2026-02-17T05:53:28.949 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 180388626442 2026-02-17T05:53:28.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:53:29.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626442 -lt 180388626442 2026-02-17T05:53:29.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:29.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-197568495624 2026-02-17T05:53:29.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:29.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:53:29.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-197568495624 2026-02-17T05:53:29.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:29.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=197568495624 2026-02-17T05:53:29.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 197568495624' 2026-02-17T05:53:29.106 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 197568495624 2026-02-17T05:53:29.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:53:29.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 197568495624 -lt 197568495624 2026-02-17T05:53:29.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:53:29.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:29.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:29.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:53:29.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:53:29.442 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:53:29.442 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:53:29.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:53:29.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:53:29.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:53:29.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:53:29.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:53:29.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:53:29.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:29.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:53:29.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:53:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:53:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:53:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:53:29.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:53:29.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:29.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:29.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:29.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:53:30.208 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:53:30.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:30.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:30.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:13.242133+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:30.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:53:31.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:53:31.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:31.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:31.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:31.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:31.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:31.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:31.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:13.242133+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:31.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:53:32.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:53:32.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:32.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:32.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:32.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:32.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:32.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:32.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:13.242133+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:32.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:53:33.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:53:33.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:33.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:33.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:33.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:33.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:33.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:33.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:13.242133+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:33.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:53:34.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:53:34.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:34.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:34.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:34.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:34.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:34.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:34.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:13.242133+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:34.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:53:35.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:53:35.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:35.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:35.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:35.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:35.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:35.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:36.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:13.242133+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:36.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:53:37.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:53:37.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:37.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:37.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:37.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:37.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:37.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:37.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:13.242133+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:37.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:53:38.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:53:38.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:53:38.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:53:38.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:53:38.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:53:38.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:53:38.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:53:38.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:31.152031+0000 '>' 2026-02-17T05:53:13.242133+0000 2026-02-17T05:53:38.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:53:38.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:53:38.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:53:38.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:53:38.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:53:38.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:53:39.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T05:53:40.061 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:53:40.062 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:53:40.062 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:53:40.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:53:40.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:53:40.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:53:40.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:53:40.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:53:40.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:53:40.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:53:40.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:53:40.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:start osd.9 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:53:40.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:53:40.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:53:40.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:53:40.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:53:40.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:53:40.619 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:40.602+0000 7f4fe888d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:40.621 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:40.606+0000 7f4fe888d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:40.621 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:40.606+0000 7f4fe888d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:40.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:53:40.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:53:40.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:53:40.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:53:40.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:53:40.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:40.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:53:40.755 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:53:40.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:40.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:40.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:41.889 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:41.874+0000 7f4fe888d8c0 -1 Falling back to public interface 2026-02-17T05:53:41.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:41.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:41.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:53:41.907 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:41.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:41.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:42.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:43.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:43.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:43.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:53:43.058 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:43.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:43.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:43.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:44.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:44.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:44.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:53:44.211 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:44.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:44.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:44.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:44.916 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:44.902+0000 7f4fe888d8c0 -1 osd.9 66 log_to_monitors true 2026-02-17T05:53:45.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:45.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:45.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:53:45.362 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:45.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:45.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:53:45.512 INFO:tasks.workunit.client.0.trial074.stderr:osd.9 up in weight 1 up_from 70 up_thru 0 down_at 67 last_clean_interval [46,66) [v2:127.0.0.1:6874/3285359990,v1:127.0.0.1:6875/3285359990] [v2:127.0.0.1:6876/3285359990,v1:127.0.0.1:6877/3285359990] exists,up eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:53:45.513 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:53:45.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:53:45.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:53:45.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:53:45.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:53:45.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:53:45.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:53:45.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:53:45.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:53:45.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:53:45.778 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:53:45.779 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:53:45.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:53:45.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:45.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:53:45.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836509 2026-02-17T05:53:45.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836509 2026-02-17T05:53:45.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509' 2026-02-17T05:53:45.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:45.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:53:45.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672987 2026-02-17T05:53:45.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672987 2026-02-17T05:53:45.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987' 2026-02-17T05:53:45.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:45.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:53:45.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509465 2026-02-17T05:53:45.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509465 2026-02-17T05:53:45.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465' 2026-02-17T05:53:45.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:45.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:53:46.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874246 2026-02-17T05:53:46.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874246 2026-02-17T05:53:46.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465 3-279172874246' 2026-02-17T05:53:46.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:46.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:53:46.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182421 2026-02-17T05:53:46.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182421 2026-02-17T05:53:46.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465 3-279172874246 4-107374182421' 2026-02-17T05:53:46.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:46.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:53:46.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018900 2026-02-17T05:53:46.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018900 2026-02-17T05:53:46.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465 3-279172874246 4-107374182421 5-128849018900' 2026-02-17T05:53:46.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:46.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:53:46.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888082 2026-02-17T05:53:46.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888082 2026-02-17T05:53:46.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465 3-279172874246 4-107374182421 5-128849018900 6-146028888082' 2026-02-17T05:53:46.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:46.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:53:46.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757264 2026-02-17T05:53:46.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757264 2026-02-17T05:53:46.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465 3-279172874246 4-107374182421 5-128849018900 6-146028888082 7-163208757264' 2026-02-17T05:53:46.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:46.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:53:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626446 2026-02-17T05:53:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626446 2026-02-17T05:53:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465 3-279172874246 4-107374182421 5-128849018900 6-146028888082 7-163208757264 8-180388626446' 2026-02-17T05:53:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:46.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:53:46.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=300647710722 2026-02-17T05:53:46.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 300647710722 2026-02-17T05:53:46.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836509 1-42949672987 2-64424509465 3-279172874246 4-107374182421 5-128849018900 6-146028888082 7-163208757264 8-180388626446 9-300647710722' 2026-02-17T05:53:46.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:46.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836509 2026-02-17T05:53:46.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:46.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:53:46.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836509 2026-02-17T05:53:46.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:46.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836509 2026-02-17T05:53:46.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-17T05:53:46.507 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836509 2026-02-17T05:53:46.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:46.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836509 2026-02-17T05:53:46.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:53:47.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:53:47.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:47.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836510 -lt 21474836509 2026-02-17T05:53:47.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:47.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672987 2026-02-17T05:53:47.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:47.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:53:47.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672987 2026-02-17T05:53:47.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672987 2026-02-17T05:53:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672987' 2026-02-17T05:53:47.819 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672987 2026-02-17T05:53:47.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:53:47.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672987 -lt 42949672987 2026-02-17T05:53:47.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:47.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509465 2026-02-17T05:53:47.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:47.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:53:47.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509465 2026-02-17T05:53:47.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:47.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509465 2026-02-17T05:53:47.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509465' 2026-02-17T05:53:47.988 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509465 2026-02-17T05:53:47.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:53:48.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509465 -lt 64424509465 2026-02-17T05:53:48.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:48.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874246 2026-02-17T05:53:48.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:48.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:53:48.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874246 2026-02-17T05:53:48.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:48.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874246 2026-02-17T05:53:48.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874246' 2026-02-17T05:53:48.145 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874246 2026-02-17T05:53:48.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:53:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874247 -lt 279172874246 2026-02-17T05:53:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:48.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182421 2026-02-17T05:53:48.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:48.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:53:48.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182421 2026-02-17T05:53:48.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:48.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182421 2026-02-17T05:53:48.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182421' 2026-02-17T05:53:48.307 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 107374182421 2026-02-17T05:53:48.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:53:48.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182422 -lt 107374182421 2026-02-17T05:53:48.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:48.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-128849018900 2026-02-17T05:53:48.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:48.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:53:48.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-128849018900 2026-02-17T05:53:48.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:48.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018900 2026-02-17T05:53:48.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 128849018900' 2026-02-17T05:53:48.476 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 128849018900 2026-02-17T05:53:48.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:53:48.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018900 -lt 128849018900 2026-02-17T05:53:48.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:48.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888082 2026-02-17T05:53:48.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:48.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:53:48.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888082 2026-02-17T05:53:48.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:48.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888082 2026-02-17T05:53:48.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888082' 2026-02-17T05:53:48.639 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 146028888082 2026-02-17T05:53:48.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:53:48.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888082 -lt 146028888082 2026-02-17T05:53:48.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:48.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757264 2026-02-17T05:53:48.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:48.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:53:48.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757264 2026-02-17T05:53:48.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:48.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757264 2026-02-17T05:53:48.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757264' 2026-02-17T05:53:48.793 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 163208757264 2026-02-17T05:53:48.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:53:48.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757264 -lt 163208757264 2026-02-17T05:53:48.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:48.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626446 2026-02-17T05:53:48.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:48.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:53:48.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626446 2026-02-17T05:53:48.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:48.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626446 2026-02-17T05:53:48.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626446' 2026-02-17T05:53:48.950 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 180388626446 2026-02-17T05:53:48.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:53:49.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626447 -lt 180388626446 2026-02-17T05:53:49.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:49.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-300647710722 2026-02-17T05:53:49.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:49.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:53:49.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-300647710722 2026-02-17T05:53:49.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:49.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=300647710722 2026-02-17T05:53:49.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 300647710722' 2026-02-17T05:53:49.115 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 300647710722 2026-02-17T05:53:49.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:53:49.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 300647710722 -lt 300647710722 2026-02-17T05:53:49.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:53:49.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:49.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:49.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:53:49.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:53:49.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:53:49.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:53:49.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:53:49.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:53:49.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:53:49.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:53:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:53:49.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:53:49.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:53:49.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:53:49.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:53:49.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:53:49.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:53:49.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:53:49.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 5 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=5 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:53:49.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:53:49.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:53:51.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:53:51.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:53:51.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:53:51.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:53:51.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:53:51.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:53:51.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:53:51.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T05:53:51.769 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:53:52.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:53:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:53:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:53:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:53:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:53:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:53:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:53:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:53:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:53:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:53:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:53:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:53:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:53:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:53:52.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:53:52.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:53:52.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:53:52.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:53:52.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:53:52.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:53:52.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:53:52.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:53:52.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:53:52.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:53:52.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:53:52.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:53:52.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:53:52.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:53:52.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:53:52.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:53:52.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:53:52.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:53:52.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:53:52.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:53:52.316 INFO:tasks.workunit.client.0.trial074.stderr:start osd.5 2026-02-17T05:53:52.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:53:52.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:53:52.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:53:52.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:53:52.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:53:52.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:53:52.324 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:52.310+0000 7f02752cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:52.325 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:52.310+0000 7f02752cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:52.326 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:52.310+0000 7f02752cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:53:52.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:53:52.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:53:52.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:53:52.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:53:52.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:53:52.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:52.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:53:52.463 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:53:52.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:52.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:53:52.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:53.561 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:53.546+0000 7f02752cc8c0 -1 Falling back to public interface 2026-02-17T05:53:53.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:53.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:53.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:53:53.613 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:53.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:53.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:53:53.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:54.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:54.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:54.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:53:54.764 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:54.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:54.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:53:54.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:55.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:55.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:55.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:53:55.916 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:55.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:55.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:53:56.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:53:56.558 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:53:56.542+0000 7f02752cc8c0 -1 osd.5 72 log_to_monitors true 2026-02-17T05:53:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:53:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:53:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:53:57.068 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:57.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:53:57.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:53:57.214 INFO:tasks.workunit.client.0.trial074.stderr:osd.5 up in weight 1 up_from 76 up_thru 76 down_at 73 last_clean_interval [30,72) [v2:127.0.0.1:6842/1671173102,v1:127.0.0.1:6843/1671173102] [v2:127.0.0.1:6844/1671173102,v1:127.0.0.1:6845/1671173102] exists,up a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:53:57.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:53:57.216 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:53:57.216 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:53:57.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:53:57.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:53:57.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:53:57.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:53:57.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:53:57.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:53:57.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:53:57.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:53:57.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:53:57.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:53:57.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:53:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:53:57.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836513 2026-02-17T05:53:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836513 2026-02-17T05:53:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513' 2026-02-17T05:53:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:53:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672990 2026-02-17T05:53:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672990 2026-02-17T05:53:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990' 2026-02-17T05:53:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:53:57.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509468 2026-02-17T05:53:57.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509468 2026-02-17T05:53:57.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468' 2026-02-17T05:53:57.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:53:57.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874250 2026-02-17T05:53:57.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874250 2026-02-17T05:53:57.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468 3-279172874250' 2026-02-17T05:53:57.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:53:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182425 2026-02-17T05:53:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182425 2026-02-17T05:53:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468 3-279172874250 4-107374182425' 2026-02-17T05:53:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:53:57.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=326417514498 2026-02-17T05:53:57.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 326417514498 2026-02-17T05:53:57.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468 3-279172874250 4-107374182425 5-326417514498' 2026-02-17T05:53:57.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:53:57.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888085 2026-02-17T05:53:57.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888085 2026-02-17T05:53:57.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468 3-279172874250 4-107374182425 5-326417514498 6-146028888085' 2026-02-17T05:53:57.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:57.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:53:58.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757267 2026-02-17T05:53:58.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757267 2026-02-17T05:53:58.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468 3-279172874250 4-107374182425 5-326417514498 6-146028888085 7-163208757267' 2026-02-17T05:53:58.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:58.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:53:58.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626450 2026-02-17T05:53:58.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626450 2026-02-17T05:53:58.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468 3-279172874250 4-107374182425 5-326417514498 6-146028888085 7-163208757267 8-180388626450' 2026-02-17T05:53:58.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:53:58.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:53:58.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=300647710725 2026-02-17T05:53:58.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 300647710725 2026-02-17T05:53:58.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672990 2-64424509468 3-279172874250 4-107374182425 5-326417514498 6-146028888085 7-163208757267 8-180388626450 9-300647710725' 2026-02-17T05:53:58.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:58.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836513 2026-02-17T05:53:58.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:58.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:53:58.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836513 2026-02-17T05:53:58.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:58.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836513 2026-02-17T05:53:58.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-17T05:53:58.209 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836513 2026-02-17T05:53:58.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:53:58.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836513 -lt 21474836513 2026-02-17T05:53:58.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:58.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672990 2026-02-17T05:53:58.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:58.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:53:58.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672990 2026-02-17T05:53:58.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:58.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672990 2026-02-17T05:53:58.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672990' 2026-02-17T05:53:58.373 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672990 2026-02-17T05:53:58.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:53:58.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672990 -lt 42949672990 2026-02-17T05:53:58.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:58.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509468 2026-02-17T05:53:58.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:58.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:53:58.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509468 2026-02-17T05:53:58.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:58.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509468 2026-02-17T05:53:58.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509468' 2026-02-17T05:53:58.538 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509468 2026-02-17T05:53:58.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:53:58.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509468 -lt 64424509468 2026-02-17T05:53:58.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:53:58.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874250 2026-02-17T05:53:58.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:53:58.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:53:58.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874250 2026-02-17T05:53:58.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:53:58.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874250 2026-02-17T05:53:58.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874250' 2026-02-17T05:53:58.698 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874250 2026-02-17T05:53:58.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:53:58.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874249 -lt 279172874250 2026-02-17T05:53:58.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:53:59.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:53:59.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:54:00.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874250 -lt 279172874250 2026-02-17T05:54:00.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:00.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182425 2026-02-17T05:54:00.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:00.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:54:00.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182425 2026-02-17T05:54:00.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:00.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182425 2026-02-17T05:54:00.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182425' 2026-02-17T05:54:00.007 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 107374182425 2026-02-17T05:54:00.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:54:00.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182425 -lt 107374182425 2026-02-17T05:54:00.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:00.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-326417514498 2026-02-17T05:54:00.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:00.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:54:00.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-326417514498 2026-02-17T05:54:00.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:00.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=326417514498 2026-02-17T05:54:00.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 326417514498' 2026-02-17T05:54:00.160 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 326417514498 2026-02-17T05:54:00.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:54:00.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514498 -lt 326417514498 2026-02-17T05:54:00.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:00.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888085 2026-02-17T05:54:00.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:00.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:54:00.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888085 2026-02-17T05:54:00.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:00.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888085 2026-02-17T05:54:00.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888085' 2026-02-17T05:54:00.320 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 146028888085 2026-02-17T05:54:00.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:54:00.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888086 -lt 146028888085 2026-02-17T05:54:00.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:00.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757267 2026-02-17T05:54:00.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:00.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:54:00.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757267 2026-02-17T05:54:00.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:00.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757267 2026-02-17T05:54:00.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757267' 2026-02-17T05:54:00.478 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 163208757267 2026-02-17T05:54:00.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:54:00.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757268 -lt 163208757267 2026-02-17T05:54:00.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:00.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626450 2026-02-17T05:54:00.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:00.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:54:00.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626450 2026-02-17T05:54:00.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:00.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626450 2026-02-17T05:54:00.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626450' 2026-02-17T05:54:00.643 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 180388626450 2026-02-17T05:54:00.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:54:00.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626450 -lt 180388626450 2026-02-17T05:54:00.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:00.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-300647710725 2026-02-17T05:54:00.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:00.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:54:00.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-300647710725 2026-02-17T05:54:00.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:00.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=300647710725 2026-02-17T05:54:00.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 300647710725' 2026-02-17T05:54:00.802 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 300647710725 2026-02-17T05:54:00.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:54:00.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 300647710725 -lt 300647710725 2026-02-17T05:54:00.957 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:54:00.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:00.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:01.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:54:01.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:54:01.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:54:01.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:54:01.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:54:01.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:54:01.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:54:01.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:54:01.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:54:01.290 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:54:01.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:01.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:01.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:54:01.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:54:01.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:54:01.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:54:01.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:54:01.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:54:01.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:54:01.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:54:01.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:54:01.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:54:01.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:54:01.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:54:01.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:01.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:01.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:01.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:01.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:01.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:54:01.918 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:54:01.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:01.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:02.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:31.152031+0000 '>' 2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:02.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:03.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:03.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:03.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:03.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:03.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:03.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:03.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:03.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:31.152031+0000 '>' 2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:03.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:04.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:04.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:04.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:04.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:04.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:04.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:04.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:04.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:31.152031+0000 '>' 2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:04.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:05.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:05.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:05.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:05.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:05.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:05.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:05.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:05.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:31.152031+0000 '>' 2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:05.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:06.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:06.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:06.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:06.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:06.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:06.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:06.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:06.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:53:31.152031+0000 '>' 2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:06.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:07.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:07.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:07.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:07.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:07.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:07.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:07.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:02.132717+0000 '>' 2026-02-17T05:53:31.152031+0000 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:54:07.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:54:07.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:54:07.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:54:07.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:54:07.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:54:09.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:54:09.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:54:09.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:54:09.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:54:09.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:54:09.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:54:09.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:54:09.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T05:54:09.462 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:54:09.462 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:54:09.462 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:54:10.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:54:10.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:54:10.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:54:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:54:10.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:start osd.9 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:54:10.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:54:10.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:54:10.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:54:10.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:54:10.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:54:10.023 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:54:10.006+0000 7f52a40e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:10.025 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:54:10.010+0000 7f52a40e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:10.025 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:54:10.010+0000 7f52a40e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:54:10.151 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:54:10.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:10.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:10.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:11.277 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:54:11.262+0000 7f52a40e48c0 -1 Falling back to public interface 2026-02-17T05:54:11.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:11.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:11.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:54:11.307 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:54:11.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:11.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:11.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:12.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:12.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:12.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:54:12.457 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:54:12.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:12.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:12.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:13.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:13.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:13.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:54:13.614 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:54:13.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:13.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:13.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:14.282 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:54:14.266+0000 7f52a40e48c0 -1 osd.9 79 log_to_monitors true 2026-02-17T05:54:14.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:14.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:14.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:54:14.770 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:54:14.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:14.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:osd.9 up in weight 1 up_from 83 up_thru 0 down_at 80 last_clean_interval [70,79) [v2:127.0.0.1:6874/3417163660,v1:127.0.0.1:6875/3417163660] [v2:127.0.0.1:6876/3417163660,v1:127.0.0.1:6877/3417163660] exists,up eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:54:14.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:54:14.921 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:54:14.921 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:54:14.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:54:14.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:54:14.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:54:15.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:54:15.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:54:15.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:54:15.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:54:15.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:54:15.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:54:15.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:54:15.172 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:54:15.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.174 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:54:15.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836517 2026-02-17T05:54:15.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836517 2026-02-17T05:54:15.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517' 2026-02-17T05:54:15.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:54:15.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672995 2026-02-17T05:54:15.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672995 2026-02-17T05:54:15.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995' 2026-02-17T05:54:15.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:54:15.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509473 2026-02-17T05:54:15.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509473 2026-02-17T05:54:15.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473' 2026-02-17T05:54:15.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:54:15.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874254 2026-02-17T05:54:15.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874254 2026-02-17T05:54:15.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473 3-279172874254' 2026-02-17T05:54:15.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:54:15.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182429 2026-02-17T05:54:15.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182429 2026-02-17T05:54:15.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473 3-279172874254 4-107374182429' 2026-02-17T05:54:15.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:54:15.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=326417514502 2026-02-17T05:54:15.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 326417514502 2026-02-17T05:54:15.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473 3-279172874254 4-107374182429 5-326417514502' 2026-02-17T05:54:15.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:54:15.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888090 2026-02-17T05:54:15.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888090 2026-02-17T05:54:15.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473 3-279172874254 4-107374182429 5-326417514502 6-146028888090' 2026-02-17T05:54:15.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:54:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757272 2026-02-17T05:54:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757272 2026-02-17T05:54:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473 3-279172874254 4-107374182429 5-326417514502 6-146028888090 7-163208757272' 2026-02-17T05:54:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:54:15.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626454 2026-02-17T05:54:15.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626454 2026-02-17T05:54:15.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473 3-279172874254 4-107374182429 5-326417514502 6-146028888090 7-163208757272 8-180388626454' 2026-02-17T05:54:15.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:15.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:54:15.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285570 2026-02-17T05:54:15.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285570 2026-02-17T05:54:15.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-42949672995 2-64424509473 3-279172874254 4-107374182429 5-326417514502 6-146028888090 7-163208757272 8-180388626454 9-356482285570' 2026-02-17T05:54:15.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:15.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836517 2026-02-17T05:54:15.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:15.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:54:15.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836517 2026-02-17T05:54:15.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:15.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836517 2026-02-17T05:54:15.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-17T05:54:15.890 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836517 2026-02-17T05:54:15.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:16.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836517 2026-02-17T05:54:16.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:16.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672995 2026-02-17T05:54:16.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:16.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:54:16.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672995 2026-02-17T05:54:16.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:16.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672995 2026-02-17T05:54:16.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672995' 2026-02-17T05:54:16.053 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 42949672995 2026-02-17T05:54:16.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:54:16.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672995 -lt 42949672995 2026-02-17T05:54:16.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:16.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509473 2026-02-17T05:54:16.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:16.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:54:16.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509473 2026-02-17T05:54:16.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:16.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509473 2026-02-17T05:54:16.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509473' 2026-02-17T05:54:16.223 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509473 2026-02-17T05:54:16.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:54:16.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509473 -lt 64424509473 2026-02-17T05:54:16.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:16.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874254 2026-02-17T05:54:16.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:16.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:54:16.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874254 2026-02-17T05:54:16.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:16.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874254 2026-02-17T05:54:16.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874254' 2026-02-17T05:54:16.387 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874254 2026-02-17T05:54:16.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:54:16.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874254 -lt 279172874254 2026-02-17T05:54:16.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:16.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182429 2026-02-17T05:54:16.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:16.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:54:16.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182429 2026-02-17T05:54:16.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:16.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182429 2026-02-17T05:54:16.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182429' 2026-02-17T05:54:16.554 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 107374182429 2026-02-17T05:54:16.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:54:16.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182429 -lt 107374182429 2026-02-17T05:54:16.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:16.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-326417514502 2026-02-17T05:54:16.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:16.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:54:16.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-326417514502 2026-02-17T05:54:16.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:16.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=326417514502 2026-02-17T05:54:16.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 326417514502' 2026-02-17T05:54:16.707 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 326417514502 2026-02-17T05:54:16.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:54:16.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514502 -lt 326417514502 2026-02-17T05:54:16.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:16.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888090 2026-02-17T05:54:16.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:16.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:54:16.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888090 2026-02-17T05:54:16.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:16.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888090 2026-02-17T05:54:16.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888090' 2026-02-17T05:54:16.867 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 146028888090 2026-02-17T05:54:16.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:54:17.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888090 -lt 146028888090 2026-02-17T05:54:17.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:17.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757272 2026-02-17T05:54:17.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:17.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:54:17.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757272 2026-02-17T05:54:17.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:17.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757272 2026-02-17T05:54:17.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757272' 2026-02-17T05:54:17.033 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 163208757272 2026-02-17T05:54:17.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:54:17.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757271 -lt 163208757272 2026-02-17T05:54:17.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:54:18.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:54:18.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:54:18.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757272 -lt 163208757272 2026-02-17T05:54:18.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:18.341 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626454 2026-02-17T05:54:18.342 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:18.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:54:18.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626454 2026-02-17T05:54:18.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:18.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626454 2026-02-17T05:54:18.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626454' 2026-02-17T05:54:18.345 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 180388626454 2026-02-17T05:54:18.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:54:18.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626455 -lt 180388626454 2026-02-17T05:54:18.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:18.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-356482285570 2026-02-17T05:54:18.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:18.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:54:18.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-356482285570 2026-02-17T05:54:18.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:18.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285570 2026-02-17T05:54:18.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 356482285570' 2026-02-17T05:54:18.498 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 356482285570 2026-02-17T05:54:18.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:54:18.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285570 -lt 356482285570 2026-02-17T05:54:18.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:54:18.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:18.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:18.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:54:18.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:54:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:54:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:54:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:54:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:54:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:54:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:54:18.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:54:18.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:54:18.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:18.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:19.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:54:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:54:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:54:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:54:19.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:54:19.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 5 9 2026-02-17T05:54:19.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=5 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=9 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 195153"' 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 195153' 2026-02-17T05:54:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 195155"' 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 195155' 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 195153 195155' 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 195153 2026-02-17T05:54:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/195156: /' 2026-02-17T05:54:19.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:54:19.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/195158: /' 2026-02-17T05:54:21.664 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.664 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:54:21.664 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:54:21.664 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:54:21.665 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING remove 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:54:21.666 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:54:21.667 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:54:21.667 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:54:21.667 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:54:21.667 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:54:21.667 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:54:21.668 INFO:tasks.workunit.client.0.trial074.stderr:195158: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:54:21.669 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:54:21.670 INFO:tasks.workunit.client.0.trial074.stderr:195158: start osd.9 2026-02-17T05:54:21.670 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:54:21.688 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.688 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:54:21.689 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:54:21.690 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:54:21.691 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:54:21.691 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:54:21.691 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:54:21.691 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:54:21.691 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:54:21.691 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:54:21.691 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:54:21.692 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:54:21.693 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:54:21.694 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:54:21.694 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:54:21.694 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:54:21.694 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:54:21.694 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:54:21.694 INFO:tasks.workunit.client.0.trial074.stderr:195156: start osd.5 2026-02-17T05:54:21.694 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: 2026-02-17T05:54:21.670+0000 7f323f7bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: 2026-02-17T05:54:21.670+0000 7f323f7bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: 2026-02-17T05:54:21.670+0000 7f323f7bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: 0 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.300 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: 1 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: 2026-02-17T05:54:23.202+0000 7f323f7bf8c0 -1 Falling back to public interface 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: 2 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: 3 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:26.301 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.344 INFO:tasks.workunit.client.0.trial074.stderr:195158: 2026-02-17T05:54:26.285+0000 7f323f7bf8c0 195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: 2026-02-17T05:54:21.694+0000 7fa554faf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: 2026-02-17T05:54:21.694+0000 7fa554faf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: 2026-02-17T05:54:21.698+0000 7fa554faf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:54:26.345 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: 0 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: 2026-02-17T05:54:22.998+0000 7fa554faf8c0 -1 Falling back to public interface 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.346 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: 1 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: 2 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.347 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.348 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.348 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:54:26.348 INFO:tasks.workunit.client.0.trial074.stderr:195156: 3 2026-02-17T05:54:26.348 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.348 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:26.348 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:26.931 INFO:tasks.workunit.client.0.trial074.stderr:195156: 2026-02-17T05:54:26.329+0000 7fa554faf8c0 -1 osd.9 84 log_to_monitors true 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: 4 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: osd.9 up in weight 1 up_from 89 up_thru 0 down_at 85 last_clean_interval [83,84) [v2:127.0.0.1:6874/3493064966,v1:127.0.0.1:6875/3493064966] [v2:127.0.0.1:6876/3493064966,v1:127.0.0.1:6877/3493064966] exists,up eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:54:26.932 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:54:26.933 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: 1 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: 2 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: 3 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: 4 2026-02-17T05:54:26.934 INFO:tasks.workunit.client.0.trial074.stderr:195158: 5 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: 6 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: 7 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: 8 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: 9' 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836521 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836521 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521' 2026-02-17T05:54:26.935 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195158: /-1 osd.5 84 log_to_monitors true 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: 4 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: osd.5 up in weight 1 up_from 89 up_thru 89 down_at 85 last_clean_interval [76,84) [v2:127.0.0.1:6842/1301504676,v1:127.0.0.1:6843/1301504676] [v2:127.0.0.1:6844/1301504676,v1:127.0.0.1:6845/1301504676] exists,up a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:54:26.949 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 1 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 2 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 3 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 4 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 5 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 6 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 7 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 8 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: 9' 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836522 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836522 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522' 2026-02-17T05:54:26.950 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.434 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672998 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672998 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998' 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509476 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509476 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476' 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.435 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874258 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874258 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476 3-279172874258' 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182433 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182433 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476 3-279172874258 4-107374182433' 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089346 2026-02-17T05:54:27.436 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089346 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476 3-279172874258 4-107374182433 5-382252089346' 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888093 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888093 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476 3-279172874258 4-107374182433 5-382252089346 6-146028888093' 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.437 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_//home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672999 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672999 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999' 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509477 2026-02-17T05:54:27.448 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509477 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477' 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874259 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874259 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477 3-279172874259' 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182434 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182434 2026-02-17T05:54:27.449 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477 3-279172874259 4-107374182434' 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089347 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089347 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477 3-279172874259 4-107374182434 5-382252089347' 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888094 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888094 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477 3-279172874259 4-107374182434 5-382252089347 6-146028888094' 2026-02-17T05:54:27.450 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:27.451 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flushpg_stats: seq=163208757275 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757275 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476 3-279172874258 4-107374182433 5-382252089346 6-146028888093 7-163208757275' 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626458 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626458 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476 3-279172874258 4-107374182433 5-382252089346 6-146028888093 7-163208757275 8-180388626458' 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089346 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089346 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-42949672998 2-64424509476 3-279172874258 4-107374182433 5-382252089346 6-146028888093 7-163208757275 8-180388626458 9-382252089346' 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836521 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836521 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836521 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.0 seq 21474836521 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836519 -lt 21474836521 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:54:28.893 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:28.894 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836521 2026-02-17T05:54:28.894 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:28.894 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672998 2026-02-17T05:54:28.894 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:28.894 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:54:28.894 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672998 2026-02-17T05:54:28.900 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/_pg_stats: seq=163208757276 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757276 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477 3-279172874259 4-107374182434 5-382252089347 6-146028888094 7-163208757276' 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626459 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626459 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477 3-279172874259 4-107374182434 5-382252089347 6-146028888094 7-163208757276 8-180388626459' 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089347 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089347 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-42949672999 2-64424509477 3-279172874259 4-107374182434 5-382252089347 6-146028888094 7-163208757276 8-180388626459 9-382252089347' 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836522 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836522 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836522 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836522' 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.0 seq 21474836522 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836519 -lt 21474836522 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836522 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672999 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:54:28.901 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672999 2026-02-17T05:54:29.358 INFO:tasks.workunit.client.0.trial074.stderr:195156: //homeubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.358 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672998 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672998' 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.1 seq 42949672998 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672999 -lt 42949672998 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509476 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:54:29.359 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509476 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509476 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509476' 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.2 seq 64424509476 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509477 -lt 64424509476 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874258 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.360 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874258 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874258 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874258' 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.3 seq 279172874258 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874259 -lt 279172874258 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182433 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:54:29.361 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182433 2026-02-17T05:54:29.362 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.362 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182433 2026-02-17T05:54:29.362 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182433' 2026-02-17T05:54:29.362 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.4 seq 107374182433 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672999 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672999' 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.1 seq 42949672999 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672999 -lt 42949672999 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509477 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509477 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509477 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509477' 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.2 seq 64424509477 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:54:29.380 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509477 -lt 64424509477 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874259 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874259 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874259 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874259' 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.3 seq 279172874259 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874259 -lt 279172874259 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182434 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182434 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182434 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182434' 2026-02-17T05:54:29.381 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.4 seq 107374182434 2026-02-17T05:54:29.978 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clon.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:54:29.978 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182434 -lt 107374182433 2026-02-17T05:54:29.978 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.978 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-382252089346 2026-02-17T05:54:29.978 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.978 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:54:29.978 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-382252089346 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089346 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 382252089346' 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.5 seq 382252089346 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089346 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888093 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888093 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888093 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888093' 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.6 seq 146028888093 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888095 -lt 146028888093 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757275 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:29.979 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757275 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757275 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757275' 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.7 seq 163208757275 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757277 -lt 163208757275 2026-02-17T05:54:29.980 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:22e.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182434 -lt 107374182434 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-382252089347 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-382252089347 2026-02-17T05:54:30.000 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089347 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 382252089347' 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.5 seq 382252089347 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089347 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888094 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888094 2026-02-17T05:54:30.001 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888094 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888094' 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.6 seq 146028888094 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888095 -lt 146028888094 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757276 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757276 2026-02-17T05:54:30.002 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:30.003 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757276 2026-02-17T05:54:30.003 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757276' 2026-02-17T05:54:30.003 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.7 seq 163208757276 2026-02-17T05:54:30.003 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:54:30.003 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757277 -lt 163208757276 2026-02-17T05:54:30.003 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:30.619 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:274: flush_pg_stats: echo 8-180388626458 2026-02-17T05:54:30.619 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:30.619 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626458 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626458 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626458' 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.8 seq 180388626458 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626459 -lt 180388626458 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-382252089346 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:30.620 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-382252089346 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089346 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 382252089346' 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: waiting osd.9 seq 382252089346 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089346 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:54:30.621 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:54:30.622 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:54:30.653 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format js274: flush_pg_stats: echo 8-180388626459 2026-02-17T05:54:30.653 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:30.653 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626459 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626459 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626459' 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.8 seq 180388626459 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626459 -lt 180388626459 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-382252089347 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:54:30.654 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-382252089347 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089347 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 382252089347' 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: waiting osd.9 seq 382252089347 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089347 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:54:30.655 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:54:30.656 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:54:30.656 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:54:30.656 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:54:30.656 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:54:30.656 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:54:30.656 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:54:30.798 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format jon status 2026-02-17T05:54:30.798 INFO:tasks.workunit.client.0.trial074.stderr:195158: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:30.798 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:54:30.798 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:54:30.798 INFO:tasks.workunit.client.0.trial074.stderr:195158: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:54:30.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:54:30.839 INFO:tasks.workunit.client.0.trial074.stderr:son status 2026-02-17T05:54:30.839 INFO:tasks.workunit.client.0.trial074.stderr:195156: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:30.839 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:54:30.839 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:54:30.839 INFO:tasks.workunit.client.0.trial074.stderr:195156: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:54:30.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:54:30.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 195155 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:54:30.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:54:30.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:54:30.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:54:30.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:54:30.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:54:30.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:30.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:30.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:30.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:31.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:31.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:54:31.282 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:54:31.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:31.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:54:31.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:31.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:54:31.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:54:31.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:31.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:31.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:31.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:31.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:31.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:31.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:02.132717+0000 '>' 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:31.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:32.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:32.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:32.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:32.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:32.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:32.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:32.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:32.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:02.132717+0000 '>' 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:32.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:33.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:33.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:33.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:33.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:33.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:33.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:33.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:33.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:02.132717+0000 '>' 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:33.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:34.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:34.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:34.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:34.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:34.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:34.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:34.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:34.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:02.132717+0000 '>' 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:34.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:35.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:35.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:35.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:35.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:35.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:35.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:35.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:36.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:02.132717+0000 '>' 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:36.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:37.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:37.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:37.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:37.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:37.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:37.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:37.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:37.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:02.132717+0000 '>' 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:37.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:54:38.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:54:38.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:54:38.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:54:38.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:54:38.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:54:38.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:54:38.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:54:38.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:32.033706+0000 '>' 2026-02-17T05:54:02.132717+0000 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 198759"' 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 198759' 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:54:38.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 198761"' 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 198761' 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 198759 198761' 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 198759 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/198762: /' 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:54:38.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/198764: /' 2026-02-17T05:54:40.517 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.517 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:54:40.518 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:54:40.518 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:54:40.518 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:54:40.518 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.518 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:54:40.518 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:54:40.519 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:54:40.519 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:54:40.519 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:54:40.519 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:54:40.519 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:54:40.519 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T05:54:40.520 INFO:tasks.workunit.client.0.trial074.stderr:198762: _ 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: hinfo_key 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: snapset 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:54:40.521 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:54:40.522 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:54:40.522 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:54:40.522 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:54:40.522 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:54:40.522 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:54:40.522 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:54:40.523 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:54:40.524 INFO:tasks.workunit.client.0.trial074.stderr:198762: start osd.5 2026-02-17T05:54:40.532 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_rec198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.532 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:54:40.532 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:54:40.532 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:54:40.532 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:54:40.532 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.532 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.533 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: _ 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: hinfo_key 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: snapset 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:54:40.534 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:54:40.535 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:54:40.535 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:54:40.535 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:54:40.536 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:54:40.537 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:54:40.538 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:54:40.538 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:54:40.538 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:54:40.538 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:54:40.538 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:54:40.538 INFO:tasks.workunit.client.0.trial074.stderr:198764: start osd.9 2026-02-17T05:54:44.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:54:44.903 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:54:44.903 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: 2026-02-17T05:54:40.521+0000 7fa03798b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: 2026-02-17T05:54:40.525+0000 7fa03798b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: 2026-02-17T05:54:40.525+0000 7fa03798b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:54:44.904 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: 0 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: 1 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: 2026-02-17T05:54:41.821+0000 7fa03798b8c0 -1 Falling back to public interface 2026-02-17T05:54:44.905 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: 2 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:54:44.906 INFO:tasks.workunit.client.0.trial074.stderr:198762: 3 2026-02-17T05:54:44.907 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:44.907 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:44.907 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:45.134 INFO:tasks.workunit.client.0.trial074.stderr:198762: 2026-02-17T05:54:44.885+0000 7faovery_ops 2026-02-17T05:54:45.134 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: 2026-02-17T05:54:40.537+0000 7f6fee8d08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: 2026-02-17T05:54:40.537+0000 7f6fee8d08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: 2026-02-17T05:54:40.537+0000 7f6fee8d08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:54:45.135 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: 0 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: 1 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:45.136 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: 2026-02-17T05:54:42.057+0000 7f6fee8d08c0 -1 Falling back to public interface 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: 2 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:45.137 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:54:45.138 INFO:tasks.workunit.client.0.trial074.stderr:198764: 3 2026-02-17T05:54:45.138 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:45.138 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:45.138 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:45.792 INFO:tasks.workunit.client.0.trial074.stderr:198764: 2026-02-17T05:54:45.117+0000 7f603798b8c0 -1 osd.5 90 log_to_monitors true 2026-02-17T05:54:45.792 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: 4 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: osd.5 up in weight 1 up_from 95 up_thru 95 down_at 91 last_clean_interval [89,90) [v2:127.0.0.1:6842/946323977,v1:127.0.0.1:6843/946323977] [v2:127.0.0.1:6844/946323977,v1:127.0.0.1:6845/946323977] exists,up a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:54:45.793 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:54:45.794 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 1 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 2 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 3 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 4 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 5 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 6 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 7 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 8 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: 9' 2026-02-17T05:54:45.795 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836526 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836526 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526' 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:19fee8d08c0 -1 osd.9 90 log_to_monitors true 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198764: 4 2026-02-17T05:54:45.796 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: osd.9 up in weight 1 up_from 97 up_thru 0 down_at 91 last_clean_interval [89,90) [v2:127.0.0.1:6874/2599464291,v1:127.0.0.1:6875/2599464291] [v2:127.0.0.1:6876/2599464291,v1:127.0.0.1:6877/2599464291] exists,up eb323775-696d-4501-887c-ef9fa5171773 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:54:45.797 INFO:tasks.workunit.client.0.trial074.stderr:198764: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: 1 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: 2 2026-02-17T05:54:45.798 INFO:tasks.workunit.client.0.trial074.stderr:198764: 3 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: 4 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: 5 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: 6 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: 7 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: 8 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: 9' 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836527 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836527 2026-02-17T05:54:45.799 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527' 2026-02-17T05:54:46.293 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673004 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673004 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004' 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509482 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509482 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482' 2026-02-17T05:54:46.294 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874264 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874264 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482 3-279172874264' 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182438 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182438 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482 3-279172874264 4-107374182438' 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:54:46.295 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893122 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893122 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482 3-279172874264 4-107374182438 5-408021893122' 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888099 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888099 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482 3-279172874264 4-107374182438 5-408021893122 6-146028888099' 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.296 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:54:46.297 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:228762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:54:46.297 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673005 2026-02-17T05:54:46.297 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673005 2026-02-17T05:54:46.297 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005' 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509483 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509483 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483' 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874263 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874263 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483 3-279172874263' 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.298 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182439 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182439 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483 3-279172874263 4-107374182439' 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893123 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893123 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483 3-279172874263 4-107374182439 5-408021893123' 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888100 2026-02-17T05:54:46.299 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888100 2026-02-17T05:54:46.300 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483 3-279172874263 4-107374182439 5-408021893123 6-146028888100' 2026-02-17T05:54:46.300 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:46.300 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:54:48.901 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265:65: flush_pg_stats: seq=163208757281 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757281 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482 3-279172874264 4-107374182438 5-408021893122 6-146028888099 7-163208757281' 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626463 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626463 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482 3-279172874264 4-107374182438 5-408021893122 6-146028888099 7-163208757281 8-180388626463' 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:48.902 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827714 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827714 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949673004 2-64424509482 3-279172874264 4-107374182438 5-408021893122 6-146028888099 7-163208757281 8-180388626463 9-416611827714' 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836527 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:54:48.903 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836527 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836527 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836527' 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.0 seq 21474836527 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836527 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836527 2026-02-17T05:54:48.904 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:54:48.905 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:54:48.905 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:48.905 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836528 -lt 21474836527 2026-02-17T05:54:48.907 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stat flush_pg_stats: seq=163208757282 2026-02-17T05:54:48.907 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757282 2026-02-17T05:54:48.907 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483 3-279172874263 4-107374182439 5-408021893123 6-146028888100 7-163208757282' 2026-02-17T05:54:48.907 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:48.907 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626464 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626464 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483 3-279172874263 4-107374182439 5-408021893123 6-146028888100 7-163208757282 8-180388626464' 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827715 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827715 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949673005 2-64424509483 3-279172874263 4-107374182439 5-408021893123 6-146028888100 7-163208757282 8-180388626464 9-416611827715' 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836526 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:48.908 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836526 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836526 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836526' 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.0 seq 21474836526 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836526 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836526 2026-02-17T05:54:48.909 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:54:48.910 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:54:48.910 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:54:48.910 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836528 -lt 21474836526 2026-02-17T05:54:49.375 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: s: for s in $seqs 2026-02-17T05:54:49.375 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673004 2026-02-17T05:54:49.375 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.375 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:54:49.375 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673004 2026-02-17T05:54:49.375 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673004 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673004' 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.1 seq 42949673004 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673005 -lt 42949673004 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509482 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509482 2026-02-17T05:54:49.376 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509482 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509482' 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.2 seq 64424509482 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509483 -lt 64424509482 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874264 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:54:49.377 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874264 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874264 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874264' 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.3 seq 279172874264 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874265 -lt 279172874264 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182438 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.378 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:54:49.384 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper for s in $seqs 2026-02-17T05:54:49.384 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673005 2026-02-17T05:54:49.384 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.384 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:54:49.384 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673005 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673005 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673005' 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.1 seq 42949673005 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673005 -lt 42949673005 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509483 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.385 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509483 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509483 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509483' 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.2 seq 64424509483 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509483 -lt 64424509483 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874263 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:54:49.386 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874263 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874263 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874263' 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.3 seq 279172874263 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874265 -lt 279172874263 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182439 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.387 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:54:49.847 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.ss.sh:2275: flush_pg_stats: echo 4-107374182438 2026-02-17T05:54:49.847 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.847 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182438 2026-02-17T05:54:49.847 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182438' 2026-02-17T05:54:49.847 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.4 seq 107374182438 2026-02-17T05:54:49.847 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182440 -lt 107374182438 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-408021893122 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-408021893122 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893122 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 408021893122' 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.5 seq 408021893122 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:54:49.848 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893123 -lt 408021893122 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888099 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888099 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888099 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888099' 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.6 seq 146028888099 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888101 -lt 146028888099 2026-02-17T05:54:49.849 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.850 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757281 2026-02-17T05:54:49.850 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.850 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:54:49.850 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757281 2026-02-17T05:54:49.850 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.850 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757281 2026-02-17T05:54:49.850 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757281' 2026-02-17T05:54:49.855 INFO:tasks.workunit.client.0.trial074.stderr:h:2275: flush_pg_stats: echo 4-107374182439 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182439 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182439' 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.4 seq 107374182439 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182440 -lt 107374182439 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-408021893123 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.856 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-408021893123 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893123 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 408021893123' 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.5 seq 408021893123 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888100 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:54:49.857 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888100 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888100 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888100' 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.6 seq 146028888100 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888101 -lt 146028888100 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757282 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757282 2026-02-17T05:54:49.858 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:49.859 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757282 2026-02-17T05:54:49.859 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757282' 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198198764: waiting osd.7 seq 163208757281 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757283 -lt 163208757281 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626463 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626463 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626463 2026-02-17T05:54:50.501 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626463' 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.8 seq 180388626463 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626465 -lt 180388626463 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827714 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827714 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827714 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827714' 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: waiting osd.9 seq 416611827714 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827715 -lt 416611827714 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:54:50.502 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: ge762: waiting osd.7 seq 163208757282 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757283 -lt 163208757282 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626464 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626464 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626464 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626464' 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.8 seq 180388626464 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626465 -lt 180388626464 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827715 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827715 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827715 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827715' 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: waiting osd.9 seq 416611827715 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:54:50.507 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:54:50.508 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:54:50.831 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_nt_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:54:50.831 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:54:50.831 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:54:50.831 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:50.831 INFO:tasks.workunit.client.0.trial074.stderr:198764: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:50.832 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:54:50.832 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:54:50.832 INFO:tasks.workunit.client.0.trial074.stderr:198764: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:54:50.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:54:50.845 INFO:tasks.workunit.client.0.trial074.stderr:um_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:54:50.845 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:54:50.845 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:54:50.845 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:54:50.845 INFO:tasks.workunit.client.0.trial074.stderr:198762: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:198762: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 198761 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:54:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:54:50.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:54:50.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 5 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=5 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 201822"' 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 201822' 2026-02-17T05:54:50.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 201824"' 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 201824' 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 201822 201824' 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 201822 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/201825: /' 2026-02-17T05:54:50.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:50.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/201827: /' 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:54:53.358 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:54:53.359 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: 201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:54:53.360 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:54:53.361 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:54:53.362 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: start osd.3 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:54:53.363 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: start osd.5 2026-02-17T05:54:53.364 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:54:57.671 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:54:57.671 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:54:57.671 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: 2026-02-17T05:54:53.362+0000 7f003e62c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: 2026-02-17T05:54:53.362+0000 7f003e62c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: 2026-02-17T05:54:53.366+0000 7f003e62c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.672 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: 0 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: 2026-02-17T05:54:54.618+0000 7f003e62c8c0 -1 Falling back to public interface 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: 1 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:54:57.673 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: 2 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: 3 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.674 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:54:57.675 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:57.913 INFO:tasks.workunit.client.0.trial074.stderr:201825: 2026-02-17T05:54:57.654+0000 7f003e62c8c0 201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: 2026-02-17T05:54:53.362+0000 7fb9493ac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: 2026-02-17T05:54:53.362+0000 7fb9493ac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: 2026-02-17T05:54:53.366+0000 7fb9493ac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:54:57.914 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: 0 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: 1 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.915 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: 2026-02-17T05:54:54.866+0000 7fb9493ac8c0 -1 Falling back to public interface 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: 2 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:57.916 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:54:57.917 INFO:tasks.workunit.client.0.trial074.stderr:201827: 3 2026-02-17T05:54:57.917 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:57.917 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:57.917 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: 2026-02-17T05:54:57.898+0000 7fb9493ac8c0 -1 osd.5 98 log_to_monitors true 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: 4 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: osd.5 up in weight 1 up_from 105 up_thru 105 down_at 99 last_clean_interval [95,98) [v2:127.0.0.1:6842/936330586,v1:127.0.0.1:6843/936330586] [v2:127.0.0.1:6844/936330586,v1:127.0.0.1:6845/936330586] exists,up a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:54:58.619 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 1 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 2 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 3 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 4 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 5 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 6 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 7 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 8 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: 9' 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:54:58.620 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836532 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836532 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532' 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201827: //-1 osd.3 98 log_to_monitors true 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201825: 4 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:54:58.621 INFO:tasks.workunit.client.0.trial074.stderr:201825: osd.3 up in weight 1 up_from 103 up_thru 105 down_at 99 last_clean_interval [65,98) [v2:127.0.0.1:6826/2855387714,v1:127.0.0.1:6827/2855387714] [v2:127.0.0.1:6828/2855387714,v1:127.0.0.1:6829/2855387714] exists,up 5f5b00fb-b073-443d-8e8a-30731fc0db9c 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:54:58.622 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 1 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 2 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 3 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 4 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 5 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 6 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 7 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 8 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: 9' 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836531 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836531 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531' 2026-02-17T05:54:58.623 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.118 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:54:59.119 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673009 2026-02-17T05:54:59.119 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673009 2026-02-17T05:54:59.119 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009' 2026-02-17T05:54:59.119 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.119 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:54:59.119 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509488 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509488 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488' 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=442381631490 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 442381631490 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488 3-442381631490' 2026-02-17T05:54:59.120 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182443 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182443 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488 3-442381631490 4-107374182443' 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566082 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566082 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488 3-442381631490 4-107374182443 5-450971566082' 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:54:59.121 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888103 2026-02-17T05:54:59.122 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888103 2026-02-17T05:54:59.122 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488 3-442381631490 4-107374182443 5-450971566082 6-146028888103' 2026-02-17T05:54:59.122 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.122 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:54:59.122 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: fluhome/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673008 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673008 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008' 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509487 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509487 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487' 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=442381631491 2026-02-17T05:54:59.123 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 442381631491 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487 3-442381631491' 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182444 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182444 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487 3-442381631491 4-107374182444' 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566083 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566083 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487 3-442381631491 4-107374182444 5-450971566083' 2026-02-17T05:54:59.124 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.125 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:54:59.125 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888104 2026-02-17T05:54:59.125 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888104 2026-02-17T05:54:59.125 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487 3-442381631491 4-107374182444 5-450971566083 6-146028888104' 2026-02-17T05:54:59.125 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:54:59.125 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:55:00.574 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_psh_pg_stats: seq=163208757286 2026-02-17T05:55:00.574 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757286 2026-02-17T05:55:00.574 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488 3-442381631490 4-107374182443 5-450971566082 6-146028888103 7-163208757286' 2026-02-17T05:55:00.574 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626468 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626468 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488 3-442381631490 4-107374182443 5-450971566082 6-146028888103 7-163208757286 8-180388626468' 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827718 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827718 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-42949673009 2-64424509488 3-442381631490 4-107374182443 5-450971566082 6-146028888103 7-163208757286 8-180388626468 9-416611827718' 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836531 2026-02-17T05:55:00.575 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836531 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836531 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836531' 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.0 seq 21474836531 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836530 -lt 21474836531 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:55:00.576 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:55:00.577 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836532 -lt 21474836531 2026-02-17T05:55:00.577 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:00.577 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673009 2026-02-17T05:55:00.577 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:00.577 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:55:00.577 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673009 2026-02-17T05:55:00.580 INFO:tasks.workunit.client.0.trial074.stderr:201825: //hog_stats: seq=163208757287 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757287 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487 3-442381631491 4-107374182444 5-450971566083 6-146028888104 7-163208757287' 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626469 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626469 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487 3-442381631491 4-107374182444 5-450971566083 6-146028888104 7-163208757287 8-180388626469' 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:00.581 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827719 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827719 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836532 1-42949673008 2-64424509487 3-442381631491 4-107374182444 5-450971566083 6-146028888104 7-163208757287 8-180388626469 9-416611827719' 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836532 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836532 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:00.582 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836532 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836532' 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.0 seq 21474836532 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836530 -lt 21474836532 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836532 -lt 21474836532 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673008 2026-02-17T05:55:00.583 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:00.584 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:55:00.584 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673008 2026-02-17T05:55:01.041 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ume/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.041 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673009 2026-02-17T05:55:01.041 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673009' 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.1 seq 42949673009 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673009 -lt 42949673009 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509488 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509488 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509488 2026-02-17T05:55:01.042 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509488' 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.2 seq 64424509488 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509488 -lt 64424509488 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-442381631490 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-442381631490 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=442381631490 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 442381631490' 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.3 seq 442381631490 2026-02-17T05:55:01.043 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 442381631491 -lt 442381631490 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182443 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182443 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182443 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182443' 2026-02-17T05:55:01.044 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.4 seq 107374182443 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clbuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673008 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673008' 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.1 seq 42949673008 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673009 -lt 42949673008 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509487 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:55:01.048 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509487 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509487 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509487' 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.2 seq 64424509487 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509488 -lt 64424509487 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-442381631491 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-442381631491 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.049 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=442381631491 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 442381631491' 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.3 seq 442381631491 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 442381631491 -lt 442381631491 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182444 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182444 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.050 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182444 2026-02-17T05:55:01.051 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182444' 2026-02-17T05:55:01.051 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.4 seq 107374182444 2026-02-17T05:55:01.660 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.one.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182444 -lt 107374182443 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-450971566082 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-450971566082 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566082 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 450971566082' 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.5 seq 450971566082 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566083 -lt 450971566082 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888103 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888103 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888103 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888103' 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.6 seq 146028888103 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888105 -lt 146028888103 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757286 2026-02-17T05:55:01.661 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757286 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757286 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757286' 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.7 seq 163208757286 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757287 -lt 163208757286 2026-02-17T05:55:01.662 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.shclient.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182444 -lt 107374182444 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-450971566083 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-450971566083 2026-02-17T05:55:01.686 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566083 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 450971566083' 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.5 seq 450971566083 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566083 -lt 450971566083 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888104 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888104 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888104 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888104' 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.6 seq 146028888104 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888105 -lt 146028888104 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757287 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:01.687 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757287 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757287 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757287' 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.7 seq 163208757287 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757287 -lt 163208757287 2026-02-17T05:55:01.688 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:02.307 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:227:2274: flush_pg_stats: echo 8-180388626468 2026-02-17T05:55:02.307 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:02.307 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:55:02.307 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626468 2026-02-17T05:55:02.307 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:02.307 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626468 2026-02-17T05:55:02.307 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626468' 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.8 seq 180388626468 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626470 -lt 180388626468 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827718 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827718 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827718 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827718' 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: waiting osd.9 seq 416611827718 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827720 -lt 416611827718 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:02.308 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:55:02.309 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format4: flush_pg_stats: echo 8-180388626469 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626469 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626469 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626469' 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.8 seq 180388626469 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:55:02.335 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626470 -lt 180388626469 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827719 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827719 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827719 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827719' 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: waiting osd.9 seq 416611827719 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827720 -lt 416611827719 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:55:02.336 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:55:02.337 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:02.491 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format jso json status 2026-02-17T05:55:02.492 INFO:tasks.workunit.client.0.trial074.stderr:201825: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:02.492 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:55:02.492 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:55:02.492 INFO:tasks.workunit.client.0.trial074.stderr:201825: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:55:02.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:55:02.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:55:02.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 201824 2026-02-17T05:55:02.512 INFO:tasks.workunit.client.0.trial074.stderr:n status 2026-02-17T05:55:02.512 INFO:tasks.workunit.client.0.trial074.stderr:201827: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:02.512 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:55:02.512 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:55:02.512 INFO:tasks.workunit.client.0.trial074.stderr:201827: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:55:02.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:55:02.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:55:02.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:55:02.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:55:02.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:55:02.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:55:02.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:02.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:02.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:02.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:02.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:02.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:55:02.960 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:02.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:02.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:02.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:03.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:32.033706+0000 '>' 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:03.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:55:04.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:55:04.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:55:04.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:55:04.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:04.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:04.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:04.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:04.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:32.033706+0000 '>' 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:04.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:55:05.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:55:05.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:55:05.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:55:05.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:05.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:05.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:05.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:05.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:32.033706+0000 '>' 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:05.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:55:06.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:55:06.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:55:06.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:55:06.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:06.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:06.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:06.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:06.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:32.033706+0000 '>' 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:06.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:55:07.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:55:07.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:55:07.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:55:07.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:07.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:07.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:07.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:07.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:32.033706+0000 '>' 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:07.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:55:08.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:55:08.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:55:08.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:55:08.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:08.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:08.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:08.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:08.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:54:32.033706+0000 '>' 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:08.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:55:09.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:55:09.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:55:09.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:55:09.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:55:09.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:55:09.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:55:09.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:55:10.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:55:03.501100+0000 '>' 2026-02-17T05:54:32.033706+0000 2026-02-17T05:55:10.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:55:10.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:55:10.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:55:10.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:55:10.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 205432"' 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 205432' 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 205433"' 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 205433' 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:55:10.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 205432 205433' 2026-02-17T05:55:10.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:55:10.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:55:10.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:55:10.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 205432 2026-02-17T05:55:10.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/205435: /' 2026-02-17T05:55:10.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:55:10.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/205437: /' 2026-02-17T05:55:12.252 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.252 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:55:12.252 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:55:12.252 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:55:12.252 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:55:12.252 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: _ 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: hinfo_key 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: snapset 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:55:12.253 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' 205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: _ 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: hinfo_key 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: snapset 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:55:12.254 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper--osd-failsafe-full-ratio=.99' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:55:12.256 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: start osd.3 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-s.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:55:12.257 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:55:12.258 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:55:12.258 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:55:12.258 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:55:12.258 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:55:12.258 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:55:12.258 INFO:tasks.workunit.client.0.trial074.stderr:205437: start osd.5 2026-02-17T05:55:16.037 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reclen=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:55:16.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: 2026-02-17T05:55:12.258+0000 7f20ad3858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: 2026-02-17T05:55:12.258+0000 7f20ad3858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: 2026-02-17T05:55:12.258+0000 7f20ad3858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:55:16.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: 0 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: 1 2026-02-17T05:55:16.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: 2026-02-17T05:55:13.770+0000 7f20ad3858c0 -1 Falling back to public interface 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: 2 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.040 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:55:16.041 INFO:tasks.workunit.client.0.trial074.stderr:205435: 3 2026-02-17T05:55:16.041 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.041 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:16.575 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T05:55:16.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:55:16.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:55:16.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:55:16.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:55:16.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:55:16.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: 2026-02-17T05:55:12.258+0000 7fef48aa48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: 2026-02-17T05:55:12.258+0000 7fef48aa48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: 2026-02-17T05:55:12.258+0000 7fef48aa48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: 0 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:55:16.576 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: 2026-02-17T05:55:13.534+0000 7fef48aa48c0 -1 Falling back to public interface 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: 1 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: 2 2026-02-17T05:55:16.577 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: 3 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:55:16.578 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:17.508 INFO:tasks.workunit.client.0.trial074.stderr:205437: 2026-02-17T05:55:16.558+0000 7fef48aa48c0 -1 osd.5 107 log_to_monitors true 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: 4 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: osd.5 up in weight 1 up_from 112 up_thru 112 down_at 108 last_clean_interval [105,107) [v2:127.0.0.1:6826/418390404,v1:127.0.0.1:6827/418390404] [v2:127.0.0.1:6828/418390404,v1:127.0.0.1:6829/418390404] exists,up a9207a4f-13df-4249-b5c5-d4f465739a8a 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:55:17.509 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: 1 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: 2 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: 3 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: 4 2026-02-17T05:55:17.510 INFO:tasks.workunit.client.0.trial074.stderr:205437: 5 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: 6 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: 7 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: 8 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: 9' 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836537 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836537 2026-02-17T05:55:17.511 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537' 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: 2026-02-17T05:55:16.806+0000 7f20ad3858c0 -1 osd.3 107 log_to_monitors true 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: 4 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: osd.3 up in weight 1 up_from 115 up_thru 115 down_at 108 last_clean_interval [103,107) [v2:127.0.0.1:6842/834951995,v1:127.0.0.1:6843/834951995] [v2:127.0.0.1:6844/834951995,v1:127.0.0.1:6845/834951995] exists,up 5f5b00fb-b073-443d-8e8a-30731fc0db9c 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:55:17.514 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:55:17.515 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 1 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 2 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 3 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 4 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 5 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 6 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 7 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 8 2026-02-17T05:55:17.516 INFO:tasks.workunit.client.0.trial074.stderr:205435: 9' 2026-02-17T05:55:17.517 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:55:17.517 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.517 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:55:17.517 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836538 2026-02-17T05:55:17.517 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836538 2026-02-17T05:55:17.517 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538' 2026-02-17T05:55:17.942 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.942 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:55:17.942 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673015 2026-02-17T05:55:17.942 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673015 2026-02-17T05:55:17.943 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015' 2026-02-17T05:55:17.943 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.943 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:55:17.943 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509493 2026-02-17T05:55:17.943 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509493 2026-02-17T05:55:17.943 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493' 2026-02-17T05:55:17.944 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.944 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:55:17.944 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=493921239043 2026-02-17T05:55:17.944 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 493921239043 2026-02-17T05:55:17.944 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493 3-493921239043' 2026-02-17T05:55:17.944 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.944 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:55:17.945 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182450 2026-02-17T05:55:17.945 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182450 2026-02-17T05:55:17.945 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493 3-493921239043 4-107374182450' 2026-02-17T05:55:17.945 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.945 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:55:17.945 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=481036337155 2026-02-17T05:55:17.945 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 481036337155 2026-02-17T05:55:17.946 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493 3-493921239043 4-107374182450 5-481036337155' 2026-02-17T05:55:17.946 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:17.946 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:55:17.946 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888110 2026-02-17T05:55:17.946 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888110 2026-02-17T05:55:17.946 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493 3-493921239043 4-107374182450 5-481036337155 6-146028888110' 2026-02-17T05:55:17.946 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: cephids 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673014 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673014 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014' 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509492 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509492 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492' 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=493921239042 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 493921239042 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492 3-493921239042' 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182449 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182449 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492 3-493921239042 4-107374182449' 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=481036337154 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 481036337154 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492 3-493921239042 4-107374182449 5-481036337154' 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888109 2026-02-17T05:55:18.013 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888109 2026-02-17T05:55:18.014 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492 3-493921239042 4-107374182449 5-481036337154 6-146028888109' 2026-02-17T05:55:18.014 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.014 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757291 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757291 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492 3-493921239042 4-107374182449 5-481036337154 6-146028888109 7-163208757291' 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626474 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626474 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492 3-493921239042 4-107374182449 5-481036337154 6-146028888109 7-163208757291 8-180388626474' 2026-02-17T05:55:18.324 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827724 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827724 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673014 2-64424509492 3-493921239042 4-107374182449 5-481036337154 6-146028888109 7-163208757291 8-180388626474 9-416611827724' 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836537 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836537 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836537 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836537' 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.0 seq 21474836537 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836538 -lt 21474836537 2026-02-17T05:55:18.325 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673014 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673014 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673014 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673014' 2026-02-17T05:55:18.326 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.1 seq 42949673014 2026-02-17T05:55:18.327 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:227 tell osd.7 flush_pg_stats 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757292 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757292 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493 3-493921239043 4-107374182450 5-481036337155 6-146028888110 7-163208757292' 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626475 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626475 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493 3-493921239043 4-107374182450 5-481036337155 6-146028888110 7-163208757292 8-180388626475' 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:55:18.328 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827725 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827725 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836538 1-42949673015 2-64424509493 3-493921239043 4-107374182450 5-481036337155 6-146028888110 7-163208757292 8-180388626475 9-416611827725' 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836538 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836538 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836538 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836538' 2026-02-17T05:55:18.329 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.0 seq 21474836538 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836538 -lt 21474836538 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673015 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673015 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673015 2026-02-17T05:55:18.330 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673015' 2026-02-17T05:55:19.950 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting 7: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:55:19.951 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673015 -lt 42949673014 2026-02-17T05:55:19.951 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:19.951 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509492 2026-02-17T05:55:19.951 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:19.951 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:55:19.952 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509492 2026-02-17T05:55:19.952 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:19.952 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509492 2026-02-17T05:55:19.952 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509492' 2026-02-17T05:55:19.952 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.2 seq 64424509492 2026-02-17T05:55:19.952 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:55:19.953 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509493 -lt 64424509492 2026-02-17T05:55:19.953 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:19.953 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-493921239042 2026-02-17T05:55:19.953 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:19.953 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:55:19.953 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-493921239042 2026-02-17T05:55:19.954 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:19.954 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=493921239042 2026-02-17T05:55:19.954 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 493921239042' 2026-02-17T05:55:19.954 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.3 seq 493921239042 2026-02-17T05:55:19.954 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:55:19.954 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 493921239042 2026-02-17T05:55:19.954 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:55:19.955 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:55:19.955 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:55:19.955 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 493921239043 -lt 493921239042 2026-02-17T05:55:19.955 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:19.955 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182449 2026-02-17T05:55:19.955 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:19.955 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:55:19.956 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182449 2026-02-17T05:55:19.956 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:19.956 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182449 2026-02-17T05:55:19.956 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 107374182449' 2026-02-17T05:55:19.956 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.4 seq 107374182449 2026-02-17T05:55:19.956 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/cosd.1 seq 42949673015 2026-02-17T05:55:19.957 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:55:19.957 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673015 -lt 42949673015 2026-02-17T05:55:19.957 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:19.957 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509493 2026-02-17T05:55:19.957 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:19.957 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:55:19.958 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509493 2026-02-17T05:55:19.958 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:19.958 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509493 2026-02-17T05:55:19.958 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509493' 2026-02-17T05:55:19.958 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.2 seq 64424509493 2026-02-17T05:55:19.958 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:55:19.958 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509493 -lt 64424509493 2026-02-17T05:55:19.959 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:19.959 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-493921239043 2026-02-17T05:55:19.959 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:19.959 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:55:19.959 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-493921239043 2026-02-17T05:55:19.960 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:19.960 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=493921239043 2026-02-17T05:55:19.960 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 493921239043' 2026-02-17T05:55:19.960 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.3 seq 493921239043 2026-02-17T05:55:19.960 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:55:19.960 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 493921239043 2026-02-17T05:55:19.960 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:55:19.961 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:55:19.961 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:55:19.961 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 493921239043 -lt 493921239043 2026-02-17T05:55:19.961 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:19.961 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-107374182450 2026-02-17T05:55:19.961 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:19.961 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:55:19.962 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-107374182450 2026-02-17T05:55:19.962 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:19.962 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182450 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'walone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182450 -lt 107374182449 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-481036337154 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-481036337154 2026-02-17T05:55:20.572 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=481036337154 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 481036337154' 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.5 seq 481036337154 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 481036337155 -lt 481036337154 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888109 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888109 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888109 2026-02-17T05:55:20.573 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888109' 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.6 seq 146028888109 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888111 -lt 146028888109 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757291 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757291 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757291 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757291' 2026-02-17T05:55:20.574 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.7 seq 163208757291 2026-02-17T05:55:20.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:55:20.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757293 -lt 163208757291 2026-02-17T05:55:20.575 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:20.584 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.siting osd.4 seq 107374182450' 2026-02-17T05:55:20.584 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.4 seq 107374182450 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182450 -lt 107374182450 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-481036337155 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-481036337155 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=481036337155 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 481036337155' 2026-02-17T05:55:20.585 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.5 seq 481036337155 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 481036337155 -lt 481036337155 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-146028888110 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-146028888110 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888110 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 146028888110' 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.6 seq 146028888110 2026-02-17T05:55:20.586 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888111 -lt 146028888110 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-163208757292 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-163208757292 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757292 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 163208757292' 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.7 seq 163208757292 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:55:20.587 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757293 -lt 163208757292 2026-02-17T05:55:21.225 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flushh:2274: flush_pg_stats: echo 8-180388626474 2026-02-17T05:55:21.225 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:21.225 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:55:21.225 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626474 2026-02-17T05:55:21.225 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:21.225 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626474 2026-02-17T05:55:21.225 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626474' 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.8 seq 180388626474 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626475 -lt 180388626474 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827724 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827724 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827724 2026-02-17T05:55:21.226 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827724' 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: waiting osd.9 seq 416611827724 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827725 -lt 416611827724 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:21.227 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:21.228 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:21.228 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:21.228 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:21.234 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --forma_pg_stats: for s in $seqs 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-180388626475 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-180388626475 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626475 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 180388626475' 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.8 seq 180388626475 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626475 -lt 180388626475 2026-02-17T05:55:21.235 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827725 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827725 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827725 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827725' 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: waiting osd.9 seq 416611827725 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827725 -lt 416611827725 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:21.236 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:21.237 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:22.036 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' -1 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=4 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:22.037 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:22.038 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=1 2026-02-17T05:55:22.039 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test 1 '!=' null 2026-02-17T05:55:22.044 INFO:tasks.workunit.client.0.trial074.stderr:20543t json status 2026-02-17T05:55:22.044 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:22.044 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:22.044 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' -1 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=4 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:22.045 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:22.046 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:22.047 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:22.048 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=1 2026-02-17T05:55:22.048 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test 1 '!=' null 2026-02-17T05:55:22.048 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1677: wait_for_clean: loop=0 2026-02-17T05:55:22.760 INFO:tasks.workunit.client.0.trial074.stderr:25: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1677: wait_for_clean: loop=0 2026-02-17T05:55:22.760 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:22.761 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:22.762 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=1 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test 1 '!=' null 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1677: wait_for_clean: loop=0 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:22.763 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:22.764 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/stan05437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:22.765 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:22.766 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=1 2026-02-17T05:55:22.767 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test 1 '!=' null 2026-02-17T05:55:22.767 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1677: wait_for_clean: loop=0 2026-02-17T05:55:22.767 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:22.767 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:22.767 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:22.767 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:22.767 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clodalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=1 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test 1 '!=' null 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1677: wait_for_clean: loop=0 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:23.402 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/cene.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=1 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test 1 '!=' null 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1677: wait_for_clean: loop=0 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:23.407 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:23.408 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:23.408 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:23.408 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:23.408 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:23.408 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:24.276 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:24.276 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:24.276 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:24.276 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:24.277 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:24.278 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:24.279 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:55:24.279 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:24.279 INFO:tasks.workunit.client.0.trial074.stderr:205437: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:24.279 INFO:tasks.workunit.client.0.trial074.stderr:2054ph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:24.279 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:24.279 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 5 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 4 '!=' 4 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T05:55:24.280 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:55:24.281 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:55:24.282 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:55:24.282 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:55:24.462 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:55:24.462 INFO:tasks.workunit.client.0.trial074.stderr:205435: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:24.462 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:55:24.463 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:55:24.463 INFO:tasks.workunit.client.0.trial074.stderr:205435: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:55:24.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:55:24.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:55:24.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 205433 2026-02-17T05:55:24.468 INFO:tasks.workunit.client.0.trial074.stderr:37: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:55:24.468 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:205437: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:55:24.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:55:24.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:55:24.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:55:24.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:55:24.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:55:24.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:55:24.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:55:24.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:55:24.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:55:24.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:55:24.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:55:24.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:55:24.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:55:24.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:55:24.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:55:24.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:55:24.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:55:24.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:55:24.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:55:24.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:55:24.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:55:24.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:55:24.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:55:24.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:55:24.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:55:24.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:55:24.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:24.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:24.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:55:24.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:55:24.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:55:24.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:55:24.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:55:24.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:55:24.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:55:24.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:55:24.862 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:55:24.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:55:24.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:55:24.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:55:24.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:55:24.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:55:24.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:55:24.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:55:24.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:55:24.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:55:24.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:55:24.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:55:24.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:55:24.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:55:24.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:24.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:24.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:55:24.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:55:24.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:55:24.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:55:24.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:55:24.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:24.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:24.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:55:24.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:55:24.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_lrc_overwrites td/osd-scrub-repair 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_corrupt_and_repair_lrc_overwrites: '[' true = true ']' 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_corrupt_and_repair_lrc_overwrites: corrupt_and_repair_lrc td/osd-scrub-repair true 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:825: corrupt_and_repair_lrc: local dir=td/osd-scrub-repair 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:826: corrupt_and_repair_lrc: local allow_overwrites=true 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:827: corrupt_and_repair_lrc: local poolname=ecpool 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:829: corrupt_and_repair_lrc: run_mon td/osd-scrub-repair a 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:55:24.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:55:24.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:55:24.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T05:55:24.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:55:24.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:24.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:24.900 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:24.900 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:24.900 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:24.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:24.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T05:55:24.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:55:24.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:55:24.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:55:24.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:55:24.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:55:24.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:55:24.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:55:24.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:55:24.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:55:24.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:55:24.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:55:24.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:55:24.990 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:55:24.990 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:55:24.991 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:55:24.992 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:55:24.992 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:24.992 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:24.992 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:55:24.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:55:24.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:55:25.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:830: corrupt_and_repair_lrc: run_mgr td/osd-scrub-repair x 2026-02-17T05:55:25.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:55:25.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:55:25.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:55:25.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:55:25.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:55:25.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:55:25.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:55:25.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:25.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:25.142 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:25.142 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:25.142 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:25.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:25.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:55:25.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:55:25.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: seq 0 9 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 0 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:55:25.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:25.168 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:55:25.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:55:25.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:55:25.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:55:25.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8fb18673-e7b2-4917-8ef9-8dbe6dbac3ce 2026-02-17T05:55:25.170 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 8fb18673-e7b2-4917-8ef9-8dbe6dbac3ce 2026-02-17T05:55:25.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 8fb18673-e7b2-4917-8ef9-8dbe6dbac3ce' 2026-02-17T05:55:25.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:55:25.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDNApRp5uILChAAPlXvvkruhmzZPlLmrqxjrA== 2026-02-17T05:55:25.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDNApRp5uILChAAPlXvvkruhmzZPlLmrqxjrA=="}' 2026-02-17T05:55:25.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8fb18673-e7b2-4917-8ef9-8dbe6dbac3ce -i td/osd-scrub-repair/0/new.json 2026-02-17T05:55:25.268 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:55:25.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:55:25.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNApRp5uILChAAPlXvvkruhmzZPlLmrqxjrA== --osd-uuid 8fb18673-e7b2-4917-8ef9-8dbe6dbac3ce 2026-02-17T05:55:25.293 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:25.278+0000 7fbd290158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:25.294 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:25.278+0000 7fbd290158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:25.295 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:25.282+0000 7fbd290158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:25.295 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:25.282+0000 7fbd290158c0 -1 bdev(0x56314a781c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:55:25.296 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:25.282+0000 7fbd290158c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:55:27.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:55:27.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:55:27.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:55:27.905 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:55:27.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:55:28.097 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:55:28.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:55:28.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:55:28.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:55:28.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:55:28.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:55:28.121 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:28.106+0000 7faf5d1808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:28.123 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:28.110+0000 7faf5d1808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:28.124 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:28.110+0000 7faf5d1808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:28.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:55:28.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:55:28.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:55:28.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:55:28.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:55:28.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:28.250 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:55:28.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:55:28.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:28.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:55:28.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:29.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:29.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:29.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:55:29.402 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:55:29.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:29.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:55:29.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:29.628 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:29.614+0000 7faf5d1808c0 -1 Falling back to public interface 2026-02-17T05:55:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:30.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:30.553 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:55:30.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:55:30.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:30.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:55:30.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:31.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:31.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:31.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:55:31.707 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:55:31.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:31.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:55:31.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:32.591 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:32.578+0000 7faf5d1808c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:55:32.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:32.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:32.863 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:55:32.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:55:32.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:32.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:55:33.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:34.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:34.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:34.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:55:34.018 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:55:34.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:34.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:55:34.172 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1020880493,v1:127.0.0.1:6803/1020880493] [v2:127.0.0.1:6804/1020880493,v1:127.0.0.1:6805/1020880493] exists,up 8fb18673-e7b2-4917-8ef9-8dbe6dbac3ce 2026-02-17T05:55:34.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:55:34.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:55:34.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 1 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:34.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:55:34.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:55:34.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:55:34.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=74652dcc-baa1-4e34-9c81-bff0bd7e1500 2026-02-17T05:55:34.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 74652dcc-baa1-4e34-9c81-bff0bd7e1500' 2026-02-17T05:55:34.177 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 74652dcc-baa1-4e34-9c81-bff0bd7e1500 2026-02-17T05:55:34.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:55:34.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDWApRpq+6rChAA82kqlZx3wcpb9WB7AGF1xQ== 2026-02-17T05:55:34.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDWApRpq+6rChAA82kqlZx3wcpb9WB7AGF1xQ=="}' 2026-02-17T05:55:34.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 74652dcc-baa1-4e34-9c81-bff0bd7e1500 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:55:34.344 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:55:34.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:55:34.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDWApRpq+6rChAA82kqlZx3wcpb9WB7AGF1xQ== --osd-uuid 74652dcc-baa1-4e34-9c81-bff0bd7e1500 2026-02-17T05:55:34.372 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:34.358+0000 7f51b0ea58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:34.373 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:34.358+0000 7f51b0ea58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:34.374 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:34.358+0000 7f51b0ea58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:34.375 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:34.362+0000 7f51b0ea58c0 -1 bdev(0x5642da329c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:55:34.375 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:34.362+0000 7f51b0ea58c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:55:36.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:55:36.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:55:36.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:55:36.644 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:55:36.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:55:36.840 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:55:36.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:55:36.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:55:36.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:55:36.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:55:36.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:55:36.864 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:36.850+0000 7f7b2abcb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:36.866 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:36.850+0000 7f7b2abcb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:36.867 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:36.854+0000 7f7b2abcb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:36.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:55:36.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:55:36.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:55:36.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:55:36.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:55:36.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:36.991 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:55:36.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:55:36.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:36.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:55:37.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:38.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:38.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:38.145 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:55:38.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:55:38.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:38.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:55:38.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:38.372 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:38.358+0000 7f7b2abcb8c0 -1 Falling back to public interface 2026-02-17T05:55:39.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:39.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:39.299 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:55:39.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:55:39.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:39.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:55:39.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:40.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:40.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:40.454 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:55:40.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:55:40.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:40.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:55:40.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:41.392 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:41.378+0000 7f7b2abcb8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:55:41.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:41.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:41.606 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:55:41.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:55:41.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:41.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:55:41.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:42.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:42.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:42.761 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:55:42.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:55:42.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:42.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2226294087,v1:127.0.0.1:6811/2226294087] [v2:127.0.0.1:6812/2226294087,v1:127.0.0.1:6813/2226294087] exists,up 74652dcc-baa1-4e34-9c81-bff0bd7e1500 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 2 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:55:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:42.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:55:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T05:55:42.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:55:42.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=29c4e674-aabe-4ffa-b772-c518a8e9d68d 2026-02-17T05:55:42.917 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 29c4e674-aabe-4ffa-b772-c518a8e9d68d 2026-02-17T05:55:42.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 29c4e674-aabe-4ffa-b772-c518a8e9d68d' 2026-02-17T05:55:42.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:55:42.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDeApRplvHFNhAA8AqbJHORn7UUtuDBSwypqg== 2026-02-17T05:55:42.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDeApRplvHFNhAA8AqbJHORn7UUtuDBSwypqg=="}' 2026-02-17T05:55:42.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 29c4e674-aabe-4ffa-b772-c518a8e9d68d -i td/osd-scrub-repair/2/new.json 2026-02-17T05:55:43.086 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:55:43.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T05:55:43.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDeApRplvHFNhAA8AqbJHORn7UUtuDBSwypqg== --osd-uuid 29c4e674-aabe-4ffa-b772-c518a8e9d68d 2026-02-17T05:55:43.115 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:43.102+0000 7fcc79e6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:43.117 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:43.102+0000 7fcc79e6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:43.118 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:43.106+0000 7fcc79e6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:43.118 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:43.106+0000 7fcc79e6a8c0 -1 bdev(0x55aa1b9d5c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T05:55:43.119 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:43.106+0000 7fcc79e6a8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T05:55:45.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T05:55:45.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:55:45.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T05:55:45.393 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T05:55:45.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:55:45.587 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T05:55:45.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T05:55:45.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:55:45.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:55:45.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:55:45.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:55:45.608 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:45.594+0000 7fcc1d0d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:45.610 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:45.594+0000 7fcc1d0d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:45.611 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:45.598+0000 7fcc1d0d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:45.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T05:55:45.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:55:45.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T05:55:45.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:55:45.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:55:45.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:45.740 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:55:45.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:55:45.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:45.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:55:45.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:46.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:46.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:46.899 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:55:46.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:55:46.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:46.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:55:47.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:47.116 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:47.102+0000 7fcc1d0d28c0 -1 Falling back to public interface 2026-02-17T05:55:48.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:48.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:48.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:55:48.054 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:55:48.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:48.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:55:48.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:49.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:49.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:49.208 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:55:49.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:55:49.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:49.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:55:49.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:50.150 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:50.138+0000 7fcc1d0d28c0 -1 osd.2 0 log_to_monitors true 2026-02-17T05:55:50.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:50.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:50.361 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:55:50.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:55:50.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:50.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:55:50.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:51.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:51.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:51.513 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:55:51.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:55:51.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:51.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T05:55:51.662 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/549803216,v1:127.0.0.1:6819/549803216] [v2:127.0.0.1:6820/549803216,v1:127.0.0.1:6821/549803216] exists,up 29c4e674-aabe-4ffa-b772-c518a8e9d68d 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 3 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:55:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:55:51.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:55:51.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:55:51.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:55:51.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:55:51.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:55:51.664 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:55:51.664 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:55:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:55:51.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:55:51.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=be445b3b-0d24-49e2-bbc9-ac29a33acfe3 2026-02-17T05:55:51.667 INFO:tasks.workunit.client.0.trial074.stdout:add osd3 be445b3b-0d24-49e2-bbc9-ac29a33acfe3 2026-02-17T05:55:51.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 be445b3b-0d24-49e2-bbc9-ac29a33acfe3' 2026-02-17T05:55:51.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:55:51.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDnApRp1XXvJxAAW4hawBTEYFt0th+Y66mfgA== 2026-02-17T05:55:51.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDnApRp1XXvJxAAW4hawBTEYFt0th+Y66mfgA=="}' 2026-02-17T05:55:51.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new be445b3b-0d24-49e2-bbc9-ac29a33acfe3 -i td/osd-scrub-repair/3/new.json 2026-02-17T05:55:51.835 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:55:51.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T05:55:51.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDnApRp1XXvJxAAW4hawBTEYFt0th+Y66mfgA== --osd-uuid be445b3b-0d24-49e2-bbc9-ac29a33acfe3 2026-02-17T05:55:51.862 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:51.850+0000 7f78934818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:51.864 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:51.850+0000 7f78934818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:51.865 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:51.850+0000 7f78934818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:51.865 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:51.850+0000 7f78934818c0 -1 bdev(0x5589b4983c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T05:55:51.865 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:51.850+0000 7f78934818c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T05:55:54.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T05:55:54.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:55:54.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T05:55:54.140 INFO:tasks.workunit.client.0.trial074.stdout:adding osd3 key to auth repository 2026-02-17T05:55:54.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:55:54.332 INFO:tasks.workunit.client.0.trial074.stdout:start osd.3 2026-02-17T05:55:54.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T05:55:54.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:55:54.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:55:54.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:55:54.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:55:54.354 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:54.342+0000 7f18e7c658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:54.356 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:54.342+0000 7f18e7c658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:54.357 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:54.342+0000 7f18e7c658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:55:54.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T05:55:54.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:55:54.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:55:54.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:55:54.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:55:54.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:54.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:55:54.487 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:55:54.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:54.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:55.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:55.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:55.636 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:55:55.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:55:55.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:55.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:55.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:55.864 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:55.850+0000 7f18e7c658c0 -1 Falling back to public interface 2026-02-17T05:55:56.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:56.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:56.792 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:55:56.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:55:56.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:56.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:56.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:57.945 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:55:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:55:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:58.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:55:58.875 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:55:58.862+0000 7f18e7c658c0 -1 osd.3 0 log_to_monitors true 2026-02-17T05:55:59.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:55:59.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:55:59.098 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:55:59.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:55:59.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:55:59.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:55:59.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:00.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:00.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:00.248 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:00.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:56:00.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:00.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:56:00.402 INFO:tasks.workunit.client.0.trial074.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/304727673,v1:127.0.0.1:6827/304727673] [v2:127.0.0.1:6828/304727673,v1:127.0.0.1:6829/304727673] exists,up be445b3b-0d24-49e2-bbc9-ac29a33acfe3 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 4 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/4 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/4' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/4/journal' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:56:00.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:56:00.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:56:00.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:56:00.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:56:00.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:56:00.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:56:00.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/4 2026-02-17T05:56:00.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:56:00.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=70da3f5f-f1d7-4991-941d-09f8de441297 2026-02-17T05:56:00.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 70da3f5f-f1d7-4991-941d-09f8de441297' 2026-02-17T05:56:00.408 INFO:tasks.workunit.client.0.trial074.stdout:add osd4 70da3f5f-f1d7-4991-941d-09f8de441297 2026-02-17T05:56:00.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:56:00.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDwApRpDS5zGBAAk7Etw4aFautTSHYTA+NS7Q== 2026-02-17T05:56:00.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDwApRpDS5zGBAAk7Etw4aFautTSHYTA+NS7Q=="}' 2026-02-17T05:56:00.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 70da3f5f-f1d7-4991-941d-09f8de441297 -i td/osd-scrub-repair/4/new.json 2026-02-17T05:56:00.573 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:56:00.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/4/new.json 2026-02-17T05:56:00.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDwApRpDS5zGBAAk7Etw4aFautTSHYTA+NS7Q== --osd-uuid 70da3f5f-f1d7-4991-941d-09f8de441297 2026-02-17T05:56:00.599 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:00.586+0000 7f39ac8f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:00.601 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:00.586+0000 7f39ac8f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:00.602 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:00.590+0000 7f39ac8f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:00.602 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:00.590+0000 7f39ac8f28c0 -1 bdev(0x556b08a81c00 td/osd-scrub-repair/4/block) open stat got: (1) Operation not permitted 2026-02-17T05:56:00.602 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:00.590+0000 7f39ac8f28c0 -1 bluestore(td/osd-scrub-repair/4) _read_fsid unparsable uuid 2026-02-17T05:56:02.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/4/keyring 2026-02-17T05:56:02.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:56:02.896 INFO:tasks.workunit.client.0.trial074.stdout:adding osd4 key to auth repository 2026-02-17T05:56:02.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T05:56:02.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:56:03.089 INFO:tasks.workunit.client.0.trial074.stdout:start osd.4 2026-02-17T05:56:03.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T05:56:03.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:56:03.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:56:03.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:56:03.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:56:03.113 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:03.098+0000 7fe93ee7b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:03.115 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:03.102+0000 7fe93ee7b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:03.116 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:03.102+0000 7fe93ee7b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:03.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T05:56:03.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:56:03.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T05:56:03.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:56:03.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:56:03.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:03.240 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:56:03.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:56:03.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:03.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:56:03.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:04.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:04.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:04.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:56:04.396 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:56:04.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:04.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:56:04.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:04.615 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:04.602+0000 7fe93ee7b8c0 -1 Falling back to public interface 2026-02-17T05:56:05.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:05.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:05.553 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:56:05.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:56:05.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:05.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:56:05.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:06.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:06.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:06.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:56:06.707 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:56:06.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:06.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:56:06.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:07.611 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:07.598+0000 7fe93ee7b8c0 -1 osd.4 0 log_to_monitors true 2026-02-17T05:56:07.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:07.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:07.859 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:56:07.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:56:07.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:07.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:56:08.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:09.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:09.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:09.017 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:09.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:56:09.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:09.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T05:56:09.168 INFO:tasks.workunit.client.0.trial074.stdout:osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3852249427,v1:127.0.0.1:6835/3852249427] [v2:127.0.0.1:6836/3852249427,v1:127.0.0.1:6837/3852249427] exists,up 70da3f5f-f1d7-4991-941d-09f8de441297 2026-02-17T05:56:09.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:56:09.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 5 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:56:09.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:56:09.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:56:09.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:56:09.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:56:09.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:56:09.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:56:09.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 d39da59b-0cad-49f9-b89a-4a273b142b64' 2026-02-17T05:56:09.176 INFO:tasks.workunit.client.0.trial074.stdout:add osd5 d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:56:09.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:56:09.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD5ApRpXfJgChAAlqjx3yStN2PYW1U2jQLJOA== 2026-02-17T05:56:09.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD5ApRpXfJgChAAlqjx3yStN2PYW1U2jQLJOA=="}' 2026-02-17T05:56:09.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d39da59b-0cad-49f9-b89a-4a273b142b64 -i td/osd-scrub-repair/5/new.json 2026-02-17T05:56:09.337 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:09.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/5/new.json 2026-02-17T05:56:09.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5ApRpXfJgChAAlqjx3yStN2PYW1U2jQLJOA== --osd-uuid d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:56:09.371 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:09.358+0000 7fb81e9b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:09.373 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:09.358+0000 7fb81e9b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:09.374 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:09.362+0000 7fb81e9b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:09.375 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:09.362+0000 7fb81e9b28c0 -1 bdev(0x5563617ddc00 td/osd-scrub-repair/5/block) open stat got: (1) Operation not permitted 2026-02-17T05:56:09.375 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:09.362+0000 7fb81e9b28c0 -1 bluestore(td/osd-scrub-repair/5) _read_fsid unparsable uuid 2026-02-17T05:56:11.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/5/keyring 2026-02-17T05:56:11.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:56:11.653 INFO:tasks.workunit.client.0.trial074.stdout:adding osd5 key to auth repository 2026-02-17T05:56:11.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T05:56:11.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:56:11.846 INFO:tasks.workunit.client.0.trial074.stdout:start osd.5 2026-02-17T05:56:11.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T05:56:11.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:56:11.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:56:11.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:56:11.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:56:11.869 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:11.854+0000 7f406ab978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:11.871 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:11.858+0000 7f406ab978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:11.872 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:11.858+0000 7f406ab978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:11.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T05:56:11.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:56:11.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:56:11.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:56:11.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:56:11.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:11.997 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:56:11.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:56:11.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:11.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:56:12.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:13.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:13.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:13.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:56:13.148 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:56:13.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:13.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:56:13.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:13.363 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:13.350+0000 7f406ab978c0 -1 Falling back to public interface 2026-02-17T05:56:14.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:14.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:14.298 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:56:14.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:56:14.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:14.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:56:14.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:15.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:15.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:15.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:56:15.452 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:56:15.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:15.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:56:15.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:16.362 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:16.350+0000 7f406ab978c0 -1 osd.5 0 log_to_monitors true 2026-02-17T05:56:16.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:16.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:16.602 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:56:16.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:56:16.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:16.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:56:16.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:17.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:17.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:17.753 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:17.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:56:17.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:17.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:56:17.904 INFO:tasks.workunit.client.0.trial074.stdout:osd.5 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3385294275,v1:127.0.0.1:6843/3385294275] [v2:127.0.0.1:6844/3385294275,v1:127.0.0.1:6845/3385294275] exists,up d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:56:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:56:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:56:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:56:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:56:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 6 2026-02-17T05:56:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:56:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/6 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/6' 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/6/journal' 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:56:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:56:17.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:56:17.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/6 2026-02-17T05:56:17.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:56:17.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2b93dbea-8e89-41a3-b6ab-1318312aaacd 2026-02-17T05:56:17.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 2b93dbea-8e89-41a3-b6ab-1318312aaacd' 2026-02-17T05:56:17.913 INFO:tasks.workunit.client.0.trial074.stdout:add osd6 2b93dbea-8e89-41a3-b6ab-1318312aaacd 2026-02-17T05:56:17.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:56:17.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQABA5RpL7VMNhAAre5DvDF8FrC+JrQINakFEw== 2026-02-17T05:56:17.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQABA5RpL7VMNhAAre5DvDF8FrC+JrQINakFEw=="}' 2026-02-17T05:56:17.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2b93dbea-8e89-41a3-b6ab-1318312aaacd -i td/osd-scrub-repair/6/new.json 2026-02-17T05:56:18.076 INFO:tasks.workunit.client.0.trial074.stdout:6 2026-02-17T05:56:18.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/6/new.json 2026-02-17T05:56:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQABA5RpL7VMNhAAre5DvDF8FrC+JrQINakFEw== --osd-uuid 2b93dbea-8e89-41a3-b6ab-1318312aaacd 2026-02-17T05:56:18.103 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:18.090+0000 7f06fb0078c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:18.104 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:18.090+0000 7f06fb0078c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:18.105 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:18.090+0000 7f06fb0078c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:18.105 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:18.090+0000 7f06fb0078c0 -1 bdev(0x55c4bc497c00 td/osd-scrub-repair/6/block) open stat got: (1) Operation not permitted 2026-02-17T05:56:18.105 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:18.090+0000 7f06fb0078c0 -1 bluestore(td/osd-scrub-repair/6) _read_fsid unparsable uuid 2026-02-17T05:56:20.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/6/keyring 2026-02-17T05:56:20.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:56:20.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T05:56:20.396 INFO:tasks.workunit.client.0.trial074.stdout:adding osd6 key to auth repository 2026-02-17T05:56:20.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:56:20.588 INFO:tasks.workunit.client.0.trial074.stdout:start osd.6 2026-02-17T05:56:20.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T05:56:20.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:56:20.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:56:20.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:56:20.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:56:20.611 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:20.598+0000 7f85cbe0d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:20.613 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:20.598+0000 7f85cbe0d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:20.614 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:20.602+0000 7f85cbe0d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:20.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T05:56:20.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:56:20.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T05:56:20.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:56:20.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:56:20.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:20.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:56:20.739 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:56:20.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:20.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:56:20.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:21.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:21.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:21.889 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:56:21.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:56:21.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:21.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:56:22.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:22.095 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:22.082+0000 7f85cbe0d8c0 -1 Falling back to public interface 2026-02-17T05:56:23.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:23.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:23.045 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:56:23.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:56:23.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:23.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:56:23.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:24.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:24.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:24.200 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:56:24.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:56:24.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:24.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:56:24.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:25.098 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:25.086+0000 7f85cbe0d8c0 -1 osd.6 0 log_to_monitors true 2026-02-17T05:56:25.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:25.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:25.352 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:56:25.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:56:25.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:25.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:56:25.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:26.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:26.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:26.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:56:26.509 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:26.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:26.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T05:56:26.658 INFO:tasks.workunit.client.0.trial074.stdout:osd.6 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/423594310,v1:127.0.0.1:6851/423594310] [v2:127.0.0.1:6852/423594310,v1:127.0.0.1:6853/423594310] exists,up 2b93dbea-8e89-41a3-b6ab-1318312aaacd 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 7 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=7 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/7 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/7' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/7/journal' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:56:26.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:56:26.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:56:26.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/7 2026-02-17T05:56:26.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:56:26.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c8b60469-8d6d-44d0-beaa-fb4783d85bc6 2026-02-17T05:56:26.663 INFO:tasks.workunit.client.0.trial074.stdout:add osd7 c8b60469-8d6d-44d0-beaa-fb4783d85bc6 2026-02-17T05:56:26.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd7 c8b60469-8d6d-44d0-beaa-fb4783d85bc6' 2026-02-17T05:56:26.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:56:26.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAKA5Rpm8S3JxAAEzhV41hjamxQMw6KlCWEFg== 2026-02-17T05:56:26.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAKA5Rpm8S3JxAAEzhV41hjamxQMw6KlCWEFg=="}' 2026-02-17T05:56:26.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c8b60469-8d6d-44d0-beaa-fb4783d85bc6 -i td/osd-scrub-repair/7/new.json 2026-02-17T05:56:26.832 INFO:tasks.workunit.client.0.trial074.stdout:7 2026-02-17T05:56:26.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/7/new.json 2026-02-17T05:56:26.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 7 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAKA5Rpm8S3JxAAEzhV41hjamxQMw6KlCWEFg== --osd-uuid c8b60469-8d6d-44d0-beaa-fb4783d85bc6 2026-02-17T05:56:26.866 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:26.854+0000 7fe0c65f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:26.867 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:26.854+0000 7fe0c65f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:26.868 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:26.854+0000 7fe0c65f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:26.869 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:26.854+0000 7fe0c65f98c0 -1 bdev(0x55b6c1ed5c00 td/osd-scrub-repair/7/block) open stat got: (1) Operation not permitted 2026-02-17T05:56:26.869 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:26.854+0000 7fe0c65f98c0 -1 bluestore(td/osd-scrub-repair/7) _read_fsid unparsable uuid 2026-02-17T05:56:29.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/7/keyring 2026-02-17T05:56:29.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:56:29.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd7 key to auth repository 2026-02-17T05:56:29.182 INFO:tasks.workunit.client.0.trial074.stdout:adding osd7 key to auth repository 2026-02-17T05:56:29.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:56:29.373 INFO:tasks.workunit.client.0.trial074.stdout:start osd.7 2026-02-17T05:56:29.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.7 2026-02-17T05:56:29.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 7 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:56:29.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:56:29.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:56:29.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:56:29.397 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:29.382+0000 7efc0b20a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:29.399 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:29.386+0000 7efc0b20a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:29.400 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:29.386+0000 7efc0b20a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:29.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 7 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=7 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:29.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:56:29.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:30.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:30.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:30.676 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:56:30.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:56:30.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:30.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:56:30.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:30.899 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:30.886+0000 7efc0b20a8c0 -1 Falling back to public interface 2026-02-17T05:56:31.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:31.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:31.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:56:31.830 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:56:31.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:31.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:56:31.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:32.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:32.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:32.984 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:56:32.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:56:32.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:32.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:56:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:33.914 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:33.902+0000 7efc0b20a8c0 -1 osd.7 0 log_to_monitors true 2026-02-17T05:56:34.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:34.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:34.134 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:56:34.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:56:34.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:34.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:56:34.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:35.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:35.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:35.290 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:35.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:56:35.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:35.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T05:56:35.442 INFO:tasks.workunit.client.0.trial074.stdout:osd.7 up in weight 1 up_from 34 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/1078534132,v1:127.0.0.1:6859/1078534132] [v2:127.0.0.1:6860/1078534132,v1:127.0.0.1:6861/1078534132] exists,up c8b60469-8d6d-44d0-beaa-fb4783d85bc6 2026-02-17T05:56:35.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:56:35.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:56:35.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:56:35.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 8 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=8 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/8 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/8' 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/8/journal' 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:56:35.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:56:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:56:35.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:56:35.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:56:35.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:56:35.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/8 2026-02-17T05:56:35.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:56:35.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=59e3c5c3-e2d7-4d44-9c39-86f5f06f5b02 2026-02-17T05:56:35.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd8 59e3c5c3-e2d7-4d44-9c39-86f5f06f5b02' 2026-02-17T05:56:35.449 INFO:tasks.workunit.client.0.trial074.stdout:add osd8 59e3c5c3-e2d7-4d44-9c39-86f5f06f5b02 2026-02-17T05:56:35.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:56:35.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQATA5Rph+jVGhAAF144VPg0fEfvf9hyWhNqRw== 2026-02-17T05:56:35.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQATA5Rph+jVGhAAF144VPg0fEfvf9hyWhNqRw=="}' 2026-02-17T05:56:35.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 59e3c5c3-e2d7-4d44-9c39-86f5f06f5b02 -i td/osd-scrub-repair/8/new.json 2026-02-17T05:56:35.611 INFO:tasks.workunit.client.0.trial074.stdout:8 2026-02-17T05:56:35.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/8/new.json 2026-02-17T05:56:35.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 8 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQATA5Rph+jVGhAAF144VPg0fEfvf9hyWhNqRw== --osd-uuid 59e3c5c3-e2d7-4d44-9c39-86f5f06f5b02 2026-02-17T05:56:35.647 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:35.634+0000 7f95e47d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:35.648 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:35.634+0000 7f95e47d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:35.649 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:35.634+0000 7f95e47d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:35.649 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:35.638+0000 7f95e47d28c0 -1 bdev(0x55e214865c00 td/osd-scrub-repair/8/block) open stat got: (1) Operation not permitted 2026-02-17T05:56:35.650 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:35.638+0000 7f95e47d28c0 -1 bluestore(td/osd-scrub-repair/8) _read_fsid unparsable uuid 2026-02-17T05:56:37.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/8/keyring 2026-02-17T05:56:37.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:56:37.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd8 key to auth repository 2026-02-17T05:56:37.929 INFO:tasks.workunit.client.0.trial074.stdout:adding osd8 key to auth repository 2026-02-17T05:56:37.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:56:38.122 INFO:tasks.workunit.client.0.trial074.stdout:start osd.8 2026-02-17T05:56:38.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.8 2026-02-17T05:56:38.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 8 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:56:38.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:56:38.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:56:38.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:56:38.145 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:38.130+0000 7f7a2e06f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:38.147 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:38.134+0000 7f7a2e06f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:38.148 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:38.134+0000 7f7a2e06f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:38.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 8 2026-02-17T05:56:38.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:56:38.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=8 2026-02-17T05:56:38.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:56:38.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:56:38.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:38.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:56:38.277 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:56:38.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:38.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:56:38.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:39.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:39.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:39.429 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:56:39.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:56:39.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:39.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:56:39.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:39.627 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:39.614+0000 7f7a2e06f8c0 -1 Falling back to public interface 2026-02-17T05:56:40.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:40.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:40.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:56:40.582 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:56:40.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:40.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:56:40.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:41.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:41.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:41.734 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:56:41.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:56:41.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:41.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:56:41.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:42.679 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:42.666+0000 7f7a2e06f8c0 -1 osd.8 0 log_to_monitors true 2026-02-17T05:56:42.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:42.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:42.886 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:56:42.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:56:42.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:42.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:56:43.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:44.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:44.042 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:44.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:44.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:56:44.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:44.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T05:56:44.193 INFO:tasks.workunit.client.0.trial074.stdout:osd.8 up in weight 1 up_from 39 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/1922034045,v1:127.0.0.1:6867/1922034045] [v2:127.0.0.1:6868/1922034045,v1:127.0.0.1:6869/1922034045] exists,up 59e3c5c3-e2d7-4d44-9c39-86f5f06f5b02 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 9 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=9 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:56:44.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:56:44.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:56:44.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:56:44.196 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:56:44.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:56:44.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:56:44.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:56:44.199 INFO:tasks.workunit.client.0.trial074.stdout:add osd9 d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:56:44.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd9 d332f160-e7d8-409f-b480-a33a4da05bef' 2026-02-17T05:56:44.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:56:44.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAcA5Rp+IYBDBAACHP2+VUzE1mNJNYlO4RCdw== 2026-02-17T05:56:44.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAcA5Rp+IYBDBAACHP2+VUzE1mNJNYlO4RCdw=="}' 2026-02-17T05:56:44.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d332f160-e7d8-409f-b480-a33a4da05bef -i td/osd-scrub-repair/9/new.json 2026-02-17T05:56:44.362 INFO:tasks.workunit.client.0.trial074.stdout:9 2026-02-17T05:56:44.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/9/new.json 2026-02-17T05:56:44.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAcA5Rp+IYBDBAACHP2+VUzE1mNJNYlO4RCdw== --osd-uuid d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:56:44.389 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:44.374+0000 7fce13d768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:44.391 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:44.378+0000 7fce13d768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:44.392 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:44.378+0000 7fce13d768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:44.392 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:44.378+0000 7fce13d768c0 -1 bdev(0x55619823bc00 td/osd-scrub-repair/9/block) open stat got: (1) Operation not permitted 2026-02-17T05:56:44.393 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:44.378+0000 7fce13d768c0 -1 bluestore(td/osd-scrub-repair/9) _read_fsid unparsable uuid 2026-02-17T05:56:46.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/9/keyring 2026-02-17T05:56:46.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:56:46.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd9 key to auth repository 2026-02-17T05:56:46.704 INFO:tasks.workunit.client.0.trial074.stdout:adding osd9 key to auth repository 2026-02-17T05:56:46.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:56:46.898 INFO:tasks.workunit.client.0.trial074.stdout:start osd.9 2026-02-17T05:56:46.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.9 2026-02-17T05:56:46.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:56:46.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:56:46.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:56:46.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:56:46.920 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:46.906+0000 7fdf099858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:46.921 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:46.910+0000 7fdf099858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:46.922 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:46.910+0000 7fdf099858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:56:47.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 9 2026-02-17T05:56:47.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:56:47.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:56:47.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:56:47.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:56:47.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:47.050 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:56:47.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:56:47.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:47.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:56:47.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:48.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:48.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:48.205 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:56:48.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:56:48.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:48.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:56:48.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:48.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:48.394+0000 7fdf099858c0 -1 Falling back to public interface 2026-02-17T05:56:49.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:49.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:49.356 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:56:49.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:56:49.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:49.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:56:49.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:50.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:50.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:50.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:56:50.504 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:56:50.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:50.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:56:50.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:51.395 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:56:51.382+0000 7fdf099858c0 -1 osd.9 0 log_to_monitors true 2026-02-17T05:56:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:56:51.654 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:56:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:51.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:56:51.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:56:52.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:56:52.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:56:52.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:56:52.809 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:56:52.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:56:52.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:56:52.957 INFO:tasks.workunit.client.0.trial074.stdout:osd.9 up in weight 1 up_from 44 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/594947959,v1:127.0.0.1:6875/594947959] [v2:127.0.0.1:6876/594947959,v1:127.0.0.1:6877/594947959] exists,up d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:56:52.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:56:52.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:56:52.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:56:52.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:834: corrupt_and_repair_lrc: create_rbd_pool 2026-02-17T05:56:52.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:56:53.105 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:56:53.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:56:53.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:56:53.311 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:56:53.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:56:54.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:56:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:835: corrupt_and_repair_lrc: wait_for_clean 2026-02-17T05:56:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:56:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:56:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:56:54.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:56:54.605 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:56:54.605 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:56:54.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:56:54.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:56:54.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:56:54.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:56:54.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:56:54.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:56:54.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:56:54.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:56:54.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:56:54.870 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:56:54.871 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:56:54.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:56:54.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:54.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:56:54.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T05:56:54.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T05:56:54.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T05:56:54.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:54.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:56:55.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705680 2026-02-17T05:56:55.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705680 2026-02-17T05:56:55.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680' 2026-02-17T05:56:55.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.012 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:56:55.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T05:56:55.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T05:56:55.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158' 2026-02-17T05:56:55.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:56:55.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411341 2026-02-17T05:56:55.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411341 2026-02-17T05:56:55.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-77309411341' 2026-02-17T05:56:55.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:56:55.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280523 2026-02-17T05:56:55.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280523 2026-02-17T05:56:55.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-77309411341 4-94489280523' 2026-02-17T05:56:55.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:56:55.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149705 2026-02-17T05:56:55.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149705 2026-02-17T05:56:55.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-77309411341 4-94489280523 5-111669149705' 2026-02-17T05:56:55.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:56:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018887 2026-02-17T05:56:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018887 2026-02-17T05:56:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-77309411341 4-94489280523 5-111669149705 6-128849018887' 2026-02-17T05:56:55.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:56:55.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888070 2026-02-17T05:56:55.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888070 2026-02-17T05:56:55.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-77309411341 4-94489280523 5-111669149705 6-128849018887 7-146028888070' 2026-02-17T05:56:55.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:56:55.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724548 2026-02-17T05:56:55.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724548 2026-02-17T05:56:55.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-77309411341 4-94489280523 5-111669149705 6-128849018887 7-146028888070 8-167503724548' 2026-02-17T05:56:55.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:56:55.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:56:55.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T05:56:55.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T05:56:55.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-77309411341 4-94489280523 5-111669149705 6-128849018887 7-146028888070 8-167503724548 9-188978561026' 2026-02-17T05:56:55.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:55.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T05:56:55.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:55.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:56:55.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T05:56:55.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:55.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T05:56:55.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T05:56:55.584 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836498 2026-02-17T05:56:55.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:56:55.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-17T05:56:55.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:55.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705680 2026-02-17T05:56:55.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:55.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:56:55.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705680 2026-02-17T05:56:55.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:55.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705680 2026-02-17T05:56:55.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-17T05:56:55.742 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705680 2026-02-17T05:56:55.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:56:55.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-17T05:56:55.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:55.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T05:56:55.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:55.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:56:55.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T05:56:55.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:55.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T05:56:55.901 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542158 2026-02-17T05:56:55.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T05:56:55.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:56:56.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-17T05:56:56.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:56.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411341 2026-02-17T05:56:56.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:56.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:56:56.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411341 2026-02-17T05:56:56.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:56.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411341 2026-02-17T05:56:56.056 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 77309411341 2026-02-17T05:56:56.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411341' 2026-02-17T05:56:56.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:56:56.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411341 -lt 77309411341 2026-02-17T05:56:56.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:56.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280523 2026-02-17T05:56:56.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:56.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:56:56.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280523 2026-02-17T05:56:56.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:56.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280523 2026-02-17T05:56:56.222 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.4 seq 94489280523 2026-02-17T05:56:56.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280523' 2026-02-17T05:56:56.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:56:56.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280523 -lt 94489280523 2026-02-17T05:56:56.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:56.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-111669149705 2026-02-17T05:56:56.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:56.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:56:56.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-111669149705 2026-02-17T05:56:56.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:56.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149705 2026-02-17T05:56:56.377 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.5 seq 111669149705 2026-02-17T05:56:56.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 111669149705' 2026-02-17T05:56:56.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:56:56.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-17T05:56:56.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:56.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018887 2026-02-17T05:56:56.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:56.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:56:56.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018887 2026-02-17T05:56:56.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:56.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018887 2026-02-17T05:56:56.544 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.6 seq 128849018887 2026-02-17T05:56:56.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018887' 2026-02-17T05:56:56.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:56:56.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018887 -lt 128849018887 2026-02-17T05:56:56.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:56.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888070 2026-02-17T05:56:56.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:56.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:56:56.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888070 2026-02-17T05:56:56.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:56.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888070 2026-02-17T05:56:56.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888070' 2026-02-17T05:56:56.710 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.7 seq 146028888070 2026-02-17T05:56:56.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:56:56.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888069 -lt 146028888070 2026-02-17T05:56:56.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:56:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:56:57.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:56:58.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888070 -lt 146028888070 2026-02-17T05:56:58.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:58.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724548 2026-02-17T05:56:58.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:58.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:56:58.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724548 2026-02-17T05:56:58.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:58.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724548 2026-02-17T05:56:58.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724548' 2026-02-17T05:56:58.028 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.8 seq 167503724548 2026-02-17T05:56:58.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:56:58.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724548 -lt 167503724548 2026-02-17T05:56:58.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:56:58.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-188978561026 2026-02-17T05:56:58.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:56:58.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:56:58.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-188978561026 2026-02-17T05:56:58.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:56:58.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T05:56:58.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 188978561026' 2026-02-17T05:56:58.185 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.9 seq 188978561026 2026-02-17T05:56:58.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:56:58.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T05:56:58.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:56:58.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:56:58.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:56:58.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:56:58.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:56:58.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:56:58.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:56:58.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:56:58.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:56:58.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:56:58.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:56:58.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:56:58.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:56:58.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:56:58.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:56:58.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:56:58.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:56:58.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:56:58.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:837: corrupt_and_repair_lrc: create_ec_pool ecpool true k=4 m=2 l=3 plugin=lrc 2026-02-17T05:56:58.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T05:56:58.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T05:56:58.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T05:56:58.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T05:56:58.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=4 m=2 l=3 plugin=lrc 2026-02-17T05:56:59.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T05:56:59.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T05:56:59.366 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T05:56:59.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:57:00.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T05:57:00.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T05:57:00.571 INFO:tasks.workunit.client.0.trial074.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T05:57:00.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T05:57:00.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:57:00.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:57:00.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:57:00.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:57:00.584 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:57:00.584 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:57:00.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:57:00.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:57:00.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:57:00.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:57:00.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:57:00.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:57:00.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:57:00.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:57:00.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:57:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:57:00.853 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:00.853 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:00.853 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:00.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:57:00.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836500 2026-02-17T05:57:00.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836500 2026-02-17T05:57:00.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500' 2026-02-17T05:57:00.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:00.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:57:00.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705682 2026-02-17T05:57:00.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705682 2026-02-17T05:57:00.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682' 2026-02-17T05:57:00.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:00.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:57:01.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542160 2026-02-17T05:57:01.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542160 2026-02-17T05:57:01.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160' 2026-02-17T05:57:01.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:01.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:57:01.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411343 2026-02-17T05:57:01.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411343 2026-02-17T05:57:01.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160 3-77309411343' 2026-02-17T05:57:01.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:01.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:57:01.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280525 2026-02-17T05:57:01.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280525 2026-02-17T05:57:01.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160 3-77309411343 4-94489280525' 2026-02-17T05:57:01.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:01.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:57:01.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149707 2026-02-17T05:57:01.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149707 2026-02-17T05:57:01.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160 3-77309411343 4-94489280525 5-111669149707' 2026-02-17T05:57:01.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:01.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:57:01.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018890 2026-02-17T05:57:01.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018890 2026-02-17T05:57:01.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160 3-77309411343 4-94489280525 5-111669149707 6-128849018890' 2026-02-17T05:57:01.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:01.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:57:01.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888072 2026-02-17T05:57:01.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888072 2026-02-17T05:57:01.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160 3-77309411343 4-94489280525 5-111669149707 6-128849018890 7-146028888072' 2026-02-17T05:57:01.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:01.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:57:01.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724550 2026-02-17T05:57:01.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724550 2026-02-17T05:57:01.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160 3-77309411343 4-94489280525 5-111669149707 6-128849018890 7-146028888072 8-167503724550' 2026-02-17T05:57:01.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:01.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:57:01.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561028 2026-02-17T05:57:01.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561028 2026-02-17T05:57:01.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542160 3-77309411343 4-94489280525 5-111669149707 6-128849018890 7-146028888072 8-167503724550 9-188978561028' 2026-02-17T05:57:01.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:01.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836500 2026-02-17T05:57:01.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:01.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:57:01.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836500 2026-02-17T05:57:01.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836500 2026-02-17T05:57:01.575 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836500 2026-02-17T05:57:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-17T05:57:01.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:57:01.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-17T05:57:01.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:01.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705682 2026-02-17T05:57:01.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:01.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:57:01.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705682 2026-02-17T05:57:01.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:01.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705682 2026-02-17T05:57:01.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-17T05:57:01.734 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705682 2026-02-17T05:57:01.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:57:01.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705682 -lt 38654705682 2026-02-17T05:57:01.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:01.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542160 2026-02-17T05:57:01.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:01.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:57:01.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542160 2026-02-17T05:57:01.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:01.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542160 2026-02-17T05:57:01.888 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542160 2026-02-17T05:57:01.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542160' 2026-02-17T05:57:01.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:57:02.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542161 -lt 60129542160 2026-02-17T05:57:02.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:02.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411343 2026-02-17T05:57:02.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:57:02.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411343 2026-02-17T05:57:02.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:02.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411343 2026-02-17T05:57:02.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411343' 2026-02-17T05:57:02.056 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 77309411343 2026-02-17T05:57:02.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:57:02.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411343 -lt 77309411343 2026-02-17T05:57:02.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:02.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280525 2026-02-17T05:57:02.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:02.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:57:02.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280525 2026-02-17T05:57:02.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280525 2026-02-17T05:57:02.218 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.4 seq 94489280525 2026-02-17T05:57:02.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280525' 2026-02-17T05:57:02.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:57:02.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280525 -lt 94489280525 2026-02-17T05:57:02.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:02.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-111669149707 2026-02-17T05:57:02.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:02.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:57:02.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-111669149707 2026-02-17T05:57:02.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:02.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149707 2026-02-17T05:57:02.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 111669149707' 2026-02-17T05:57:02.375 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.5 seq 111669149707 2026-02-17T05:57:02.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:57:02.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149707 -lt 111669149707 2026-02-17T05:57:02.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:02.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018890 2026-02-17T05:57:02.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:02.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:57:02.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018890 2026-02-17T05:57:02.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:02.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018890 2026-02-17T05:57:02.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018890' 2026-02-17T05:57:02.531 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.6 seq 128849018890 2026-02-17T05:57:02.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:57:02.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018890 -lt 128849018890 2026-02-17T05:57:02.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:02.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888072 2026-02-17T05:57:02.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:02.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:57:02.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888072 2026-02-17T05:57:02.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:02.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888072 2026-02-17T05:57:02.692 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.7 seq 146028888072 2026-02-17T05:57:02.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888072' 2026-02-17T05:57:02.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:57:02.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888071 -lt 146028888072 2026-02-17T05:57:02.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:57:03.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:57:03.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:57:04.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888072 -lt 146028888072 2026-02-17T05:57:04.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:04.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724550 2026-02-17T05:57:04.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:04.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:57:04.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724550 2026-02-17T05:57:04.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:04.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724550 2026-02-17T05:57:04.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724550' 2026-02-17T05:57:04.006 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.8 seq 167503724550 2026-02-17T05:57:04.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:57:04.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-17T05:57:04.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:04.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-188978561028 2026-02-17T05:57:04.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:04.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:57:04.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-188978561028 2026-02-17T05:57:04.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:04.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561028 2026-02-17T05:57:04.161 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.9 seq 188978561028 2026-02-17T05:57:04.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 188978561028' 2026-02-17T05:57:04.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:57:04.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 188978561028 2026-02-17T05:57:04.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:57:04.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:04.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:04.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:57:04.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:57:04.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:57:04.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:57:04.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:57:04.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:57:04.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:57:04.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:57:04.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:57:04.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:57:04.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:04.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:04.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:57:04.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:57:04.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:57:04.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T05:57:04.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:838: corrupt_and_repair_lrc: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T05:57:04.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T05:57:04.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T05:57:05.030 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T05:57:05.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T05:57:05.233 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T05:57:05.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T05:57:05.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T05:57:05.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:57:05.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T05:57:05.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T05:57:05.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:57:05.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:57:05.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:57:05.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T05:57:05.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T05:57:05.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T05:57:05.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T05:57:05.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T05:57:05.423 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:57:05.424 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T05:57:05.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T05:57:05.576 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:57:05.576 INFO:tasks.workunit.client.0.trial074.stderr:9 2026-02-17T05:57:05.577 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:57:05.577 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:57:05.577 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:05.577 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:05.577 INFO:tasks.workunit.client.0.trial074.stderr:7' 2026-02-17T05:57:05.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 9 0 6 2 1 7 2026-02-17T05:57:05.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('5' '9' '0' '6' '2' '1' '7') 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=5 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=9 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:57:05.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:57:05.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:57:05.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:57:05.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:57:05.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:57:06.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:57:06.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:57:06.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:57:06.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:57:06.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:57:06.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:57:06.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:57:06.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:57:07.521 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:57:08.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:57:08.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:57:08.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:57:08.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:57:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:start osd.3 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:57:08.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:57:08.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:57:08.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:57:08.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:57:08.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:57:08.075 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:08.062+0000 7f87b4e588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:08.077 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:08.066+0000 7f87b4e588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:08.078 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:08.066+0000 7f87b4e588c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:57:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:57:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:57:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:57:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:57:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:57:08.211 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:57:08.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:08.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:57:08.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:09.339 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:09.326+0000 7f87b4e588c0 -1 Falling back to public interface 2026-02-17T05:57:09.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:57:09.366 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:57:09.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:10.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:10.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:10.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:57:10.517 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:10.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:10.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:57:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:11.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:11.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:11.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:57:11.669 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:11.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:11.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:57:11.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:12.327 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:12.314+0000 7f87b4e588c0 -1 osd.3 61 log_to_monitors true 2026-02-17T05:57:12.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:12.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:12.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:57:12.823 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:12.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:12.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:57:12.971 INFO:tasks.workunit.client.0.trial074.stderr:osd.3 up in weight 1 up_from 65 up_thru 65 down_at 62 last_clean_interval [18,61) [v2:127.0.0.1:6826/2315000582,v1:127.0.0.1:6827/2315000582] [v2:127.0.0.1:6828/2315000582,v1:127.0.0.1:6829/2315000582] exists,up be445b3b-0d24-49e2-bbc9-ac29a33acfe3 2026-02-17T05:57:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:57:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:57:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:57:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:57:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:57:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:57:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:57:12.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:57:12.973 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:57:12.973 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:57:12.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:57:12.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:57:12.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:57:13.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:57:13.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:57:13.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:57:13.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:57:13.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:57:13.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:57:13.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:57:13.239 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:13.239 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:13.239 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:13.239 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:13.239 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:57:13.239 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:57:13.240 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:57:13.240 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:57:13.240 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:57:13.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:57:13.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:57:13.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836503 2026-02-17T05:57:13.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836503 2026-02-17T05:57:13.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503' 2026-02-17T05:57:13.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:57:13.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705686 2026-02-17T05:57:13.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705686 2026-02-17T05:57:13.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686' 2026-02-17T05:57:13.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:57:13.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542164 2026-02-17T05:57:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542164 2026-02-17T05:57:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164' 2026-02-17T05:57:13.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:57:13.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874242 2026-02-17T05:57:13.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874242 2026-02-17T05:57:13.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164 3-279172874242' 2026-02-17T05:57:13.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:57:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280528 2026-02-17T05:57:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280528 2026-02-17T05:57:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164 3-279172874242 4-94489280528' 2026-02-17T05:57:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:57:13.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149711 2026-02-17T05:57:13.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149711 2026-02-17T05:57:13.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164 3-279172874242 4-94489280528 5-111669149711' 2026-02-17T05:57:13.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:57:13.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018893 2026-02-17T05:57:13.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018893 2026-02-17T05:57:13.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164 3-279172874242 4-94489280528 5-111669149711 6-128849018893' 2026-02-17T05:57:13.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:57:13.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888075 2026-02-17T05:57:13.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888075 2026-02-17T05:57:13.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164 3-279172874242 4-94489280528 5-111669149711 6-128849018893 7-146028888075' 2026-02-17T05:57:13.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:57:13.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724554 2026-02-17T05:57:13.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724554 2026-02-17T05:57:13.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164 3-279172874242 4-94489280528 5-111669149711 6-128849018893 7-146028888075 8-167503724554' 2026-02-17T05:57:13.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:13.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:57:13.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561032 2026-02-17T05:57:13.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561032 2026-02-17T05:57:13.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-60129542164 3-279172874242 4-94489280528 5-111669149711 6-128849018893 7-146028888075 8-167503724554 9-188978561032' 2026-02-17T05:57:13.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:13.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836503 2026-02-17T05:57:13.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:13.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:57:13.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836503 2026-02-17T05:57:13.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:13.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836503 2026-02-17T05:57:13.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-17T05:57:13.965 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836503 2026-02-17T05:57:13.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:57:14.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836503 -lt 21474836503 2026-02-17T05:57:14.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:14.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705686 2026-02-17T05:57:14.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:14.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:57:14.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705686 2026-02-17T05:57:14.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:14.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705686 2026-02-17T05:57:14.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705686' 2026-02-17T05:57:14.121 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705686 2026-02-17T05:57:14.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:57:14.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705686 -lt 38654705686 2026-02-17T05:57:14.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:14.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542164 2026-02-17T05:57:14.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:14.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:57:14.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542164 2026-02-17T05:57:14.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:14.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542164 2026-02-17T05:57:14.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542164' 2026-02-17T05:57:14.280 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542164 2026-02-17T05:57:14.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:57:14.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542163 -lt 60129542164 2026-02-17T05:57:14.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:57:15.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:57:15.441 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:57:15.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542164 -lt 60129542164 2026-02-17T05:57:15.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:15.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874242 2026-02-17T05:57:15.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:15.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:57:15.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874242 2026-02-17T05:57:15.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:15.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874242 2026-02-17T05:57:15.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874242' 2026-02-17T05:57:15.596 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874242 2026-02-17T05:57:15.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:57:15.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874242 -lt 279172874242 2026-02-17T05:57:15.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:15.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280528 2026-02-17T05:57:15.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:15.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:57:15.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280528 2026-02-17T05:57:15.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:15.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280528 2026-02-17T05:57:15.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280528' 2026-02-17T05:57:15.763 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 94489280528 2026-02-17T05:57:15.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:57:15.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280529 -lt 94489280528 2026-02-17T05:57:15.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:15.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-111669149711 2026-02-17T05:57:15.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:15.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:57:15.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-111669149711 2026-02-17T05:57:15.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:15.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149711 2026-02-17T05:57:15.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 111669149711' 2026-02-17T05:57:15.928 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 111669149711 2026-02-17T05:57:15.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:57:16.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149711 -lt 111669149711 2026-02-17T05:57:16.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:16.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018893 2026-02-17T05:57:16.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:16.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:57:16.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018893 2026-02-17T05:57:16.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:16.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018893 2026-02-17T05:57:16.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018893' 2026-02-17T05:57:16.091 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 128849018893 2026-02-17T05:57:16.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:57:16.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018893 -lt 128849018893 2026-02-17T05:57:16.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:16.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888075 2026-02-17T05:57:16.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:16.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:57:16.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888075 2026-02-17T05:57:16.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:16.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888075 2026-02-17T05:57:16.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888075' 2026-02-17T05:57:16.258 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 146028888075 2026-02-17T05:57:16.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:57:16.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888076 -lt 146028888075 2026-02-17T05:57:16.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:16.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724554 2026-02-17T05:57:16.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:16.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:57:16.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724554 2026-02-17T05:57:16.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:16.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724554 2026-02-17T05:57:16.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724554' 2026-02-17T05:57:16.414 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 167503724554 2026-02-17T05:57:16.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:57:16.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724554 -lt 167503724554 2026-02-17T05:57:16.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:16.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-188978561032 2026-02-17T05:57:16.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:16.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:57:16.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-188978561032 2026-02-17T05:57:16.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:16.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561032 2026-02-17T05:57:16.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 188978561032' 2026-02-17T05:57:16.571 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 188978561032 2026-02-17T05:57:16.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:57:16.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561032 -lt 188978561032 2026-02-17T05:57:16.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:57:16.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:16.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:16.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:57:16.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:57:16.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:57:16.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:57:16.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:57:16.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:57:16.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:57:16.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:57:17.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:57:17.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:57:17.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:17.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:57:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:57:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:57:17.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:57:17.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:57:17.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:57:17.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:57:17.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:57:17.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:57:17.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:57:17.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:57:17.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:57:17.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:17.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:17.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:17.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:17.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:17.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:57:17.677 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:57:17.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:17.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:17.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:17.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:56:59.355604+0000 '>' 2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:17.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:18.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:18.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:18.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:18.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:18.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:18.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:56:59.355604+0000 '>' 2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:18.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:19.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:19.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:19.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:19.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:19.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:19.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:19.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:20.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:56:59.355604+0000 '>' 2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:20.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:21.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:21.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:21.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:21.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:21.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:21.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:21.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:21.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:56:59.355604+0000 '>' 2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:21.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:22.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:22.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:22.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:22.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:22.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:22.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:22.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:22.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:56:59.355604+0000 '>' 2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:22.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:23.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:23.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:23.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:23.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:23.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:23.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:23.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:57:18.179081+0000 '>' 2026-02-17T05:56:59.355604+0000 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:57:23.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:57:23.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:57:23.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:57:24.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:57:24.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:57:24.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:57:24.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:57:24.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:57:24.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:57:24.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:57:24.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T05:57:25.213 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:57:25.213 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:57:25.213 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:57:25.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:57:25.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:57:25.759 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:57:25.759 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:57:25.760 INFO:tasks.workunit.client.0.trial074.stderr:start osd.9 2026-02-17T05:57:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:57:25.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:57:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:57:25.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:57:25.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:57:25.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:57:25.778 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:25.766+0000 7f1a30ebd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:25.779 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:25.766+0000 7f1a30ebd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:25.780 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:25.766+0000 7f1a30ebd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:57:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:25.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:26.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:27.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:27.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:27.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:57:27.068 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:27.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:27.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:27.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:27.254 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:27.242+0000 7f1a30ebd8c0 -1 Falling back to public interface 2026-02-17T05:57:28.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:28.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:28.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:57:28.223 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:28.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:28.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:28.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:29.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:29.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:29.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:57:29.377 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:29.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:29.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:29.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:30.287 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:30.274+0000 7f1a30ebd8c0 -1 osd.9 67 log_to_monitors true 2026-02-17T05:57:30.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:30.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:30.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:57:30.529 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:30.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:30.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:30.680 INFO:tasks.workunit.client.0.trial074.stderr:osd.9 up in weight 1 up_from 71 up_thru 0 down_at 68 last_clean_interval [44,67) [v2:127.0.0.1:6874/81330880,v1:127.0.0.1:6875/81330880] [v2:127.0.0.1:6876/81330880,v1:127.0.0.1:6877/81330880] exists,up d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:57:30.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:57:30.682 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:57:30.683 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:57:30.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:57:30.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:57:30.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:57:30.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:57:30.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:57:30.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:57:30.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:57:30.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:57:30.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:57:30.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:57:30.935 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:30.935 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:30.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:57:31.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836508 2026-02-17T05:57:31.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836508 2026-02-17T05:57:31.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508' 2026-02-17T05:57:31.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:57:31.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705690 2026-02-17T05:57:31.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705690 2026-02-17T05:57:31.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690' 2026-02-17T05:57:31.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:57:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542168 2026-02-17T05:57:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542168 2026-02-17T05:57:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168' 2026-02-17T05:57:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:57:31.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874246 2026-02-17T05:57:31.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874246 2026-02-17T05:57:31.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168 3-279172874246' 2026-02-17T05:57:31.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:57:31.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280533 2026-02-17T05:57:31.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280533 2026-02-17T05:57:31.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168 3-279172874246 4-94489280533' 2026-02-17T05:57:31.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:57:31.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149715 2026-02-17T05:57:31.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149715 2026-02-17T05:57:31.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168 3-279172874246 4-94489280533 5-111669149715' 2026-02-17T05:57:31.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:57:31.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018898 2026-02-17T05:57:31.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018898 2026-02-17T05:57:31.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168 3-279172874246 4-94489280533 5-111669149715 6-128849018898' 2026-02-17T05:57:31.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:57:31.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888080 2026-02-17T05:57:31.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888080 2026-02-17T05:57:31.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168 3-279172874246 4-94489280533 5-111669149715 6-128849018898 7-146028888080' 2026-02-17T05:57:31.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:57:31.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724558 2026-02-17T05:57:31.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724558 2026-02-17T05:57:31.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168 3-279172874246 4-94489280533 5-111669149715 6-128849018898 7-146028888080 8-167503724558' 2026-02-17T05:57:31.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:31.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:57:31.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678018 2026-02-17T05:57:31.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678018 2026-02-17T05:57:31.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-60129542168 3-279172874246 4-94489280533 5-111669149715 6-128849018898 7-146028888080 8-167503724558 9-304942678018' 2026-02-17T05:57:31.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:31.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836508 2026-02-17T05:57:31.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:31.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:57:31.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836508 2026-02-17T05:57:31.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:31.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836508 2026-02-17T05:57:31.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-17T05:57:31.658 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836508 2026-02-17T05:57:31.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:57:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-17T05:57:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:31.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705690 2026-02-17T05:57:31.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:57:31.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705690 2026-02-17T05:57:31.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:31.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705690 2026-02-17T05:57:31.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705690' 2026-02-17T05:57:31.826 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705690 2026-02-17T05:57:31.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:57:31.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705690 -lt 38654705690 2026-02-17T05:57:31.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:31.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542168 2026-02-17T05:57:31.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:31.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:57:31.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542168 2026-02-17T05:57:31.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:31.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542168 2026-02-17T05:57:31.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542168' 2026-02-17T05:57:31.982 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542168 2026-02-17T05:57:31.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:57:32.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542169 -lt 60129542168 2026-02-17T05:57:32.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:32.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874246 2026-02-17T05:57:32.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:32.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:57:32.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874246 2026-02-17T05:57:32.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:32.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874246 2026-02-17T05:57:32.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874246' 2026-02-17T05:57:32.137 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874246 2026-02-17T05:57:32.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:57:32.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874246 -lt 279172874246 2026-02-17T05:57:32.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:32.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280533 2026-02-17T05:57:32.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:32.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:57:32.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280533 2026-02-17T05:57:32.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:32.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280533 2026-02-17T05:57:32.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280533' 2026-02-17T05:57:32.290 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 94489280533 2026-02-17T05:57:32.290 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:57:32.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280533 -lt 94489280533 2026-02-17T05:57:32.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:32.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-111669149715 2026-02-17T05:57:32.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:32.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:57:32.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-111669149715 2026-02-17T05:57:32.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:32.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149715 2026-02-17T05:57:32.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 111669149715' 2026-02-17T05:57:32.447 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 111669149715 2026-02-17T05:57:32.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:57:32.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149715 -lt 111669149715 2026-02-17T05:57:32.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:32.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018898 2026-02-17T05:57:32.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:32.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:57:32.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018898 2026-02-17T05:57:32.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:32.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018898 2026-02-17T05:57:32.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018898' 2026-02-17T05:57:32.615 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 128849018898 2026-02-17T05:57:32.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:57:32.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018897 -lt 128849018898 2026-02-17T05:57:32.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:57:33.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:57:33.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:57:33.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018898 -lt 128849018898 2026-02-17T05:57:33.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:33.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888080 2026-02-17T05:57:33.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:33.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:57:33.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888080 2026-02-17T05:57:33.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:33.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888080 2026-02-17T05:57:33.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888080' 2026-02-17T05:57:33.933 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 146028888080 2026-02-17T05:57:33.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:57:34.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888080 -lt 146028888080 2026-02-17T05:57:34.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:34.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724558 2026-02-17T05:57:34.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:34.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:57:34.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724558 2026-02-17T05:57:34.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:34.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724558 2026-02-17T05:57:34.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724558' 2026-02-17T05:57:34.100 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 167503724558 2026-02-17T05:57:34.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:57:34.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724558 -lt 167503724558 2026-02-17T05:57:34.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:34.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-304942678018 2026-02-17T05:57:34.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:34.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:57:34.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-304942678018 2026-02-17T05:57:34.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:34.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678018 2026-02-17T05:57:34.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 304942678018' 2026-02-17T05:57:34.267 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 304942678018 2026-02-17T05:57:34.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:57:34.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678018 -lt 304942678018 2026-02-17T05:57:34.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:57:34.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:34.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:34.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:57:34.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:57:34.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:57:34.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:57:34.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:57:34.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:57:34.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:57:34.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:57:34.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:57:34.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:57:34.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:34.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:34.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:57:34.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:57:34.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:57:34.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:57:34.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:57:34.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 5 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=5 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:57:34.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:57:34.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:57:36.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:57:36.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:57:36.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:57:36.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:57:36.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:57:36.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:57:36.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:57:36.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T05:57:36.922 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:57:37.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:57:37.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:57:37.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:57:37.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:37.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:57:37.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:57:37.480 INFO:tasks.workunit.client.0.trial074.stderr:start osd.5 2026-02-17T05:57:37.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:57:37.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:57:37.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:57:37.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:57:37.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:57:37.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:57:37.495 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:37.482+0000 7f977fdce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:37.497 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:37.486+0000 7f977fdce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:37.498 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:37.486+0000 7f977fdce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:37.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:57:37.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:57:37.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:57:37.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:57:37.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:57:37.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:37.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:57:37.633 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:57:37.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:37.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:57:37.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:38.767 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:38.754+0000 7f977fdce8c0 -1 Falling back to public interface 2026-02-17T05:57:38.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:38.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:38.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:57:38.787 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:38.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:38.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:57:38.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:39.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:39.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:39.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:57:39.935 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:39.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:39.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:57:40.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:41.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:41.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:41.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:57:41.089 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:41.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:41.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:57:41.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:41.726 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:41.714+0000 7f977fdce8c0 -1 osd.5 72 log_to_monitors true 2026-02-17T05:57:42.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:42.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:42.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:57:42.239 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:42.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:42.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:osd.5 up in weight 1 up_from 77 up_thru 77 down_at 73 last_clean_interval [26,72) [v2:127.0.0.1:6842/2204232643,v1:127.0.0.1:6843/2204232643] [v2:127.0.0.1:6844/2204232643,v1:127.0.0.1:6845/2204232643] exists,up d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:57:42.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:57:42.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:57:42.385 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:57:42.385 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:57:42.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:57:42.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:57:42.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:57:42.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:57:42.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:57:42.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:57:42.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:57:42.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:57:42.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:57:42.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:57:42.649 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:42.649 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:42.649 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:42.649 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:42.649 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:57:42.650 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:57:42.650 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:57:42.650 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:57:42.650 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:57:42.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:57:42.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:42.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:57:42.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836511 2026-02-17T05:57:42.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836511 2026-02-17T05:57:42.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511' 2026-02-17T05:57:42.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:42.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:57:42.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705694 2026-02-17T05:57:42.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705694 2026-02-17T05:57:42.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694' 2026-02-17T05:57:42.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:42.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:57:42.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542172 2026-02-17T05:57:42.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542172 2026-02-17T05:57:42.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172' 2026-02-17T05:57:42.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:42.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:57:42.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874250 2026-02-17T05:57:42.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874250 2026-02-17T05:57:42.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172 3-279172874250' 2026-02-17T05:57:42.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:42.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:57:43.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280536 2026-02-17T05:57:43.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280536 2026-02-17T05:57:43.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172 3-279172874250 4-94489280536' 2026-02-17T05:57:43.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:43.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:57:43.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481794 2026-02-17T05:57:43.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481794 2026-02-17T05:57:43.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172 3-279172874250 4-94489280536 5-330712481794' 2026-02-17T05:57:43.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:43.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:57:43.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018901 2026-02-17T05:57:43.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018901 2026-02-17T05:57:43.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172 3-279172874250 4-94489280536 5-330712481794 6-128849018901' 2026-02-17T05:57:43.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:43.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:57:43.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888083 2026-02-17T05:57:43.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888083 2026-02-17T05:57:43.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172 3-279172874250 4-94489280536 5-330712481794 6-128849018901 7-146028888083' 2026-02-17T05:57:43.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:43.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:57:43.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724561 2026-02-17T05:57:43.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724561 2026-02-17T05:57:43.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172 3-279172874250 4-94489280536 5-330712481794 6-128849018901 7-146028888083 8-167503724561' 2026-02-17T05:57:43.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:43.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:57:43.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678021 2026-02-17T05:57:43.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678021 2026-02-17T05:57:43.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-60129542172 3-279172874250 4-94489280536 5-330712481794 6-128849018901 7-146028888083 8-167503724561 9-304942678021' 2026-02-17T05:57:43.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:43.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836511 2026-02-17T05:57:43.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:43.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:57:43.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836511 2026-02-17T05:57:43.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:43.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836511 2026-02-17T05:57:43.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836511' 2026-02-17T05:57:43.379 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836511 2026-02-17T05:57:43.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:57:43.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836511 2026-02-17T05:57:43.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:43.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705694 2026-02-17T05:57:43.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:43.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:57:43.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705694 2026-02-17T05:57:43.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:43.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705694 2026-02-17T05:57:43.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705694' 2026-02-17T05:57:43.536 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705694 2026-02-17T05:57:43.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:57:43.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705694 -lt 38654705694 2026-02-17T05:57:43.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:43.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542172 2026-02-17T05:57:43.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:43.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:57:43.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542172 2026-02-17T05:57:43.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:43.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542172 2026-02-17T05:57:43.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542172' 2026-02-17T05:57:43.702 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542172 2026-02-17T05:57:43.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:57:43.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542172 -lt 60129542172 2026-02-17T05:57:43.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:43.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874250 2026-02-17T05:57:43.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:43.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:57:43.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874250 2026-02-17T05:57:43.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:43.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874250 2026-02-17T05:57:43.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874250' 2026-02-17T05:57:43.857 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874250 2026-02-17T05:57:43.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:57:44.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874250 -lt 279172874250 2026-02-17T05:57:44.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:44.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280536 2026-02-17T05:57:44.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:44.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:57:44.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280536 2026-02-17T05:57:44.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:44.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280536 2026-02-17T05:57:44.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280536' 2026-02-17T05:57:44.018 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 94489280536 2026-02-17T05:57:44.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:57:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280536 -lt 94489280536 2026-02-17T05:57:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:44.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-330712481794 2026-02-17T05:57:44.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:44.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:57:44.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-330712481794 2026-02-17T05:57:44.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:44.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481794 2026-02-17T05:57:44.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 330712481794' 2026-02-17T05:57:44.174 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 330712481794 2026-02-17T05:57:44.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:57:44.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481794 -lt 330712481794 2026-02-17T05:57:44.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:44.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018901 2026-02-17T05:57:44.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:44.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:57:44.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018901 2026-02-17T05:57:44.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018901 2026-02-17T05:57:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018901' 2026-02-17T05:57:44.333 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 128849018901 2026-02-17T05:57:44.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:57:44.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018901 -lt 128849018901 2026-02-17T05:57:44.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:44.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888083 2026-02-17T05:57:44.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:44.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:57:44.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888083 2026-02-17T05:57:44.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:44.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888083 2026-02-17T05:57:44.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888083' 2026-02-17T05:57:44.487 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 146028888083 2026-02-17T05:57:44.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:57:44.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888083 -lt 146028888083 2026-02-17T05:57:44.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:44.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724561 2026-02-17T05:57:44.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:44.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:57:44.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724561 2026-02-17T05:57:44.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:44.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724561 2026-02-17T05:57:44.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724561' 2026-02-17T05:57:44.651 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 167503724561 2026-02-17T05:57:44.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:57:44.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724561 -lt 167503724561 2026-02-17T05:57:44.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:44.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-304942678021 2026-02-17T05:57:44.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:44.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:57:44.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-304942678021 2026-02-17T05:57:44.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:44.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678021 2026-02-17T05:57:44.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 304942678021' 2026-02-17T05:57:44.805 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 304942678021 2026-02-17T05:57:44.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:57:44.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678021 -lt 304942678021 2026-02-17T05:57:44.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:57:44.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:44.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:45.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:57:45.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:57:45.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:57:45.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:57:45.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:57:45.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:57:45.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:57:45.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:57:45.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:57:45.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:57:45.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:57:45.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:57:45.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:57:45.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:57:45.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:57:45.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T05:57:45.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:57:45.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:57:45.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:57:45.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:57:45.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T05:57:45.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T05:57:45.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:57:45.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:57:45.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:45.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:45.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:45.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:45.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:57:18.179081+0000 2026-02-17T05:57:45.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:57:45.915 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:57:18.179081+0000 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:57:18.179081+0000 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:45.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:45.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:45.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:46.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:57:18.179081+0000 '>' 2026-02-17T05:57:18.179081+0000 2026-02-17T05:57:46.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:47.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:47.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:47.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:47.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:47.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:47.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:47.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:47.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:57:18.179081+0000 '>' 2026-02-17T05:57:18.179081+0000 2026-02-17T05:57:47.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:48.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:48.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:48.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:48.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:48.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:48.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:48.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:48.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:57:18.179081+0000 '>' 2026-02-17T05:57:18.179081+0000 2026-02-17T05:57:48.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:57:49.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:57:49.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:57:49.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:57:49.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:57:49.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:57:49.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:57:49.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:57:49.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:57:46.331719+0000 '>' 2026-02-17T05:57:18.179081+0000 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:57:49.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:57:49.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:57:49.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:57:49.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:57:49.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:57:49.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:57:49.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:57:49.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:57:50.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:57:50.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:57:50.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:57:50.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:57:50.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:57:50.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:57:50.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:57:50.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T05:57:51.162 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:57:51.163 INFO:tasks.workunit.client.0.trial074.stdout:hinfo_key 2026-02-17T05:57:51.163 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:57:51.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:57:51.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:57:51.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:57:51.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:57:51.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:57:51.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:57:51.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:57:51.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:57:51.700 INFO:tasks.workunit.client.0.trial074.stderr:start osd.9 2026-02-17T05:57:51.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:57:51.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:57:51.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:57:51.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:57:51.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:57:51.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:57:51.722 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:51.710+0000 7f86fa69a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:51.723 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:51.710+0000 7f86fa69a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:51.724 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:51.710+0000 7f86fa69a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:57:51.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:52.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:52.950 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:52.938+0000 7f86fa69a8c0 -1 Falling back to public interface 2026-02-17T05:57:53.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:53.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:53.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:57:53.008 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:53.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:53.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:53.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:54.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:54.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:54.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:57:54.157 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:54.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:54.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:54.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:55.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:55.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:55.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:57:55.309 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:55.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:55.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:55.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:57:55.977 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:57:55.966+0000 7f86fa69a8c0 -1 osd.9 79 log_to_monitors true 2026-02-17T05:57:56.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:57:56.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:57:56.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:57:56.459 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:56.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:57:56.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:57:56.606 INFO:tasks.workunit.client.0.trial074.stderr:osd.9 up in weight 1 up_from 83 up_thru 0 down_at 80 last_clean_interval [71,79) [v2:127.0.0.1:6874/2409940810,v1:127.0.0.1:6875/2409940810] [v2:127.0.0.1:6876/2409940810,v1:127.0.0.1:6877/2409940810] exists,up d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:57:56.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:57:56.608 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:57:56.608 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:57:56.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:57:56.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:57:56.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:57:56.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:57:56.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:57:56.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:57:56.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:57:56.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:57:56.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:57:56.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:57:56.880 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:57:56.880 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:57:56.880 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:57:56.880 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:57:56.880 INFO:tasks.workunit.client.0.trial074.stderr:5 2026-02-17T05:57:56.880 INFO:tasks.workunit.client.0.trial074.stderr:6 2026-02-17T05:57:56.880 INFO:tasks.workunit.client.0.trial074.stderr:7 2026-02-17T05:57:56.881 INFO:tasks.workunit.client.0.trial074.stderr:8 2026-02-17T05:57:56.881 INFO:tasks.workunit.client.0.trial074.stderr:9' 2026-02-17T05:57:56.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:57:56.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:56.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:57:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836515 2026-02-17T05:57:56.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836515 2026-02-17T05:57:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515' 2026-02-17T05:57:56.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:56.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:57:57.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705697 2026-02-17T05:57:57.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705697 2026-02-17T05:57:57.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697' 2026-02-17T05:57:57.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:57:57.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542176 2026-02-17T05:57:57.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542176 2026-02-17T05:57:57.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176' 2026-02-17T05:57:57.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:57:57.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874253 2026-02-17T05:57:57.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874253 2026-02-17T05:57:57.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253' 2026-02-17T05:57:57.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:57:57.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280540 2026-02-17T05:57:57.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280540 2026-02-17T05:57:57.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-94489280540' 2026-02-17T05:57:57.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:57:57.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481798 2026-02-17T05:57:57.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481798 2026-02-17T05:57:57.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-94489280540 5-330712481798' 2026-02-17T05:57:57.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:57:57.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018905 2026-02-17T05:57:57.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018905 2026-02-17T05:57:57.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-94489280540 5-330712481798 6-128849018905' 2026-02-17T05:57:57.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:57:57.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888087 2026-02-17T05:57:57.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888087 2026-02-17T05:57:57.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-94489280540 5-330712481798 6-128849018905 7-146028888087' 2026-02-17T05:57:57.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:57:57.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724565 2026-02-17T05:57:57.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724565 2026-02-17T05:57:57.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-94489280540 5-330712481798 6-128849018905 7-146028888087 8-167503724565' 2026-02-17T05:57:57.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:57:57.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:57:57.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285570 2026-02-17T05:57:57.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285570 2026-02-17T05:57:57.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-94489280540 5-330712481798 6-128849018905 7-146028888087 8-167503724565 9-356482285570' 2026-02-17T05:57:57.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:57.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836515 2026-02-17T05:57:57.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:57.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:57:57.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836515 2026-02-17T05:57:57.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:57.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836515 2026-02-17T05:57:57.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836515' 2026-02-17T05:57:57.606 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836515 2026-02-17T05:57:57.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:57:57.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836515 -lt 21474836515 2026-02-17T05:57:57.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:57.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705697 2026-02-17T05:57:57.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:57.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:57:57.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705697 2026-02-17T05:57:57.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:57.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705697 2026-02-17T05:57:57.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705697' 2026-02-17T05:57:57.774 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705697 2026-02-17T05:57:57.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:57:57.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705697 -lt 38654705697 2026-02-17T05:57:57.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:57.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542176 2026-02-17T05:57:57.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:57.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:57:57.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542176 2026-02-17T05:57:57.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:57.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542176 2026-02-17T05:57:57.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542176' 2026-02-17T05:57:57.929 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542176 2026-02-17T05:57:57.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:57:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542176 -lt 60129542176 2026-02-17T05:57:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:58.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874253 2026-02-17T05:57:58.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:57:58.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874253 2026-02-17T05:57:58.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:58.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874253 2026-02-17T05:57:58.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874253' 2026-02-17T05:57:58.088 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.3 seq 279172874253 2026-02-17T05:57:58.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:57:58.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874253 -lt 279172874253 2026-02-17T05:57:58.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:58.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280540 2026-02-17T05:57:58.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:58.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:57:58.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280540 2026-02-17T05:57:58.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:58.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280540 2026-02-17T05:57:58.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280540' 2026-02-17T05:57:58.253 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.4 seq 94489280540 2026-02-17T05:57:58.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:57:58.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280540 -lt 94489280540 2026-02-17T05:57:58.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:58.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-330712481798 2026-02-17T05:57:58.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:58.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:57:58.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-330712481798 2026-02-17T05:57:58.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:58.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481798 2026-02-17T05:57:58.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 330712481798' 2026-02-17T05:57:58.423 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.5 seq 330712481798 2026-02-17T05:57:58.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:57:58.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481798 -lt 330712481798 2026-02-17T05:57:58.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:58.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018905 2026-02-17T05:57:58.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:58.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:57:58.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018905 2026-02-17T05:57:58.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:58.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018905 2026-02-17T05:57:58.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018905' 2026-02-17T05:57:58.580 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.6 seq 128849018905 2026-02-17T05:57:58.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:57:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018905 -lt 128849018905 2026-02-17T05:57:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:57:58.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888087 2026-02-17T05:57:58.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:57:58.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:57:58.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888087 2026-02-17T05:57:58.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:57:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888087 2026-02-17T05:57:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888087' 2026-02-17T05:57:58.749 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.7 seq 146028888087 2026-02-17T05:57:58.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:57:58.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888086 -lt 146028888087 2026-02-17T05:57:58.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:57:59.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:57:59.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888087 -lt 146028888087 2026-02-17T05:58:00.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:00.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724565 2026-02-17T05:58:00.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:00.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:00.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724565 2026-02-17T05:58:00.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:00.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724565 2026-02-17T05:58:00.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724565' 2026-02-17T05:58:00.058 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.8 seq 167503724565 2026-02-17T05:58:00.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:00.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724566 -lt 167503724565 2026-02-17T05:58:00.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:00.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-356482285570 2026-02-17T05:58:00.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:00.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:00.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-356482285570 2026-02-17T05:58:00.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:00.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285570 2026-02-17T05:58:00.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 356482285570' 2026-02-17T05:58:00.222 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.9 seq 356482285570 2026-02-17T05:58:00.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:00.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285570 -lt 356482285570 2026-02-17T05:58:00.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:00.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:00.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:00.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:00.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:00.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:00.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:00.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:00.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:00.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:00.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:00.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:00.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:00.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:00.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:58:00.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:58:00.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 5 9 2026-02-17T05:58:00.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=5 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=9 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 233279"' 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 233279' 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 233281"' 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 233281' 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 233279 233281' 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:00.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 233279 2026-02-17T05:58:00.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:00.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/233282: /' 2026-02-17T05:58:00.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:58:00.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/233284: /' 2026-02-17T05:58:03.406 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.406 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:03.406 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:58:03.406 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:03.406 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:03.407 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:58:03.408 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:58:03.409 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:03.410 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:03.410 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:03.410 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:03.411 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:03.411 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:03.411 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:58:03.411 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:58:03.411 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:03.411 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:03.411 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:03.412 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:03.413 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:03.414 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:58:03.414 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:58:03.414 INFO:tasks.workunit.client.0.trial074.stderr:233282: start osd.5 2026-02-17T05:58:03.414 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:58:03.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:03.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:58:03.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:03.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:58:03.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:03.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:03.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T05:58:03.421 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.421 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:03.421 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:58:03.421 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:03.421 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:58:03.421 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING remove 2026-02-17T05:58:03.421 INFO:tasks.workunit.client.0.trial074.stderr:233284: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:03.422 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:03.423 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:03.423 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:03.423 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:58:03.423 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:58:03.423 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:03.423 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:03.424 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: start osd.9 2026-02-17T05:58:03.425 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:58:07.738 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:58:07.738 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: 2026-02-17T05:58:03.422+0000 7f0fa23738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: 2026-02-17T05:58:03.422+0000 7f0fa23738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: 2026-02-17T05:58:03.426+0000 7f0fa23738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:07.739 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: 0 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: 2026-02-17T05:58:04.706+0000 7f0fa23738c0 -1 Falling back to public interface 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.740 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: 1 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: 2 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:07.741 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:07.742 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.742 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:07.742 INFO:tasks.workunit.client.0.trial074.stderr:233284: 3 2026-02-17T05:58:07.742 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.742 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:07.742 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233284: 2026-02-17T05:58:07.726+0000 7f0fa23738c0 233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: 2026-02-17T05:58:03.422+0000 7f446dac48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: 2026-02-17T05:58:03.422+0000 7f446dac48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: 2026-02-17T05:58:03.422+0000 7f446dac48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:07.751 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: 0 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:07.752 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: 2026-02-17T05:58:04.694+0000 7f446dac48c0 -1 Falling back to public interface 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: 1 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:07.753 INFO:tasks.workunit.client.0.trial074.stderr:233282: 2 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: 3 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:07.754 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:08.671 INFO:tasks.workunit.client.0.trial074.stderr:233282: 2026-02-17T05:58:07.738+0000 7f446dac48c0 -1 osd.9 84 log_to_monitors true 2026-02-17T05:58:08.671 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: 4 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: osd.9 up in weight 1 up_from 89 up_thru 0 down_at 85 last_clean_interval [83,84) [v2:127.0.0.1:6874/4265176532,v1:127.0.0.1:6875/4265176532] [v2:127.0.0.1:6876/4265176532,v1:127.0.0.1:6877/4265176532] exists,up d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:08.672 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:08.673 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: 1 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: 2 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: 3 2026-02-17T05:58:08.674 INFO:tasks.workunit.client.0.trial074.stderr:233284: 4 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: 5 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: 6 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: 7 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: 8 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: 9' 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836518 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836518 2026-02-17T05:58:08.675 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518' 2026-02-17T05:58:08.676 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:08.676 INFO:tasks.workunit.client.0.trial074.stderr:233284: /-1 osd.5 84 log_to_monitors true 2026-02-17T05:58:08.676 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:08.676 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:08.676 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: 4 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: osd.5 up in weight 1 up_from 89 up_thru 89 down_at 85 last_clean_interval [77,84) [v2:127.0.0.1:6842/3027352470,v1:127.0.0.1:6843/3027352470] [v2:127.0.0.1:6844/3027352470,v1:127.0.0.1:6845/3027352470] exists,up d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:08.677 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:08.678 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 1 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 2 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 3 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 4 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 5 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 6 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 7 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 8 2026-02-17T05:58:08.679 INFO:tasks.workunit.client.0.trial074.stderr:233282: 9' 2026-02-17T05:58:08.680 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:08.680 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:08.680 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:08.680 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836519 2026-02-17T05:58:08.680 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836519 2026-02-17T05:58:08.680 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519' 2026-02-17T05:58:08.680 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705702 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705702 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702' 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542180 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542180 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180' 2026-02-17T05:58:09.173 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874258 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874258 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180 3-279172874258' 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280545 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280545 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180 3-279172874258 4-94489280545' 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089347 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089347 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180 3-279172874258 4-94489280545 5-382252089347' 2026-02-17T05:58:09.174 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018909 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018909 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180 3-279172874258 4-94489280545 5-382252089347 6-128849018909' 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_st//home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705701 2026-02-17T05:58:09.175 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705701 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701' 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542179 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542179 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179' 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874257 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874257 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257' 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280544 2026-02-17T05:58:09.176 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280544 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-94489280544' 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089346 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089346 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-94489280544 5-382252089346' 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018908 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018908 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-94489280544 5-382252089346 6-128849018908' 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.177 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:58:09.475 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_sats: seq=146028888090 2026-02-17T05:58:09.475 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888090 2026-02-17T05:58:09.475 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180 3-279172874258 4-94489280545 5-382252089347 6-128849018909 7-146028888090' 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724569 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724569 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180 3-279172874258 4-94489280545 5-382252089347 6-128849018909 7-146028888090 8-167503724569' 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089346 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089346 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-38654705702 2-60129542180 3-279172874258 4-94489280545 5-382252089347 6-128849018909 7-146028888090 8-167503724569 9-382252089346' 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836518 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836518 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836518 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.0 seq 21474836518 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:09.476 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836518 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705702 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705702 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705702 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705702' 2026-02-17T05:58:09.477 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.1 seq 38654705702 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ctats: seq=146028888091 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888091 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-94489280544 5-382252089346 6-128849018908 7-146028888091' 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724570 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724570 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-94489280544 5-382252089346 6-128849018908 7-146028888091 8-167503724570' 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089347 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089347 2026-02-17T05:58:09.480 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-94489280544 5-382252089346 6-128849018908 7-146028888091 8-167503724570 9-382252089347' 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836519 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836519 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836519 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836519' 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.0 seq 21474836519 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836519 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705701 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705701 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705701 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-17T05:58:09.481 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.1 seq 38654705701 2026-02-17T05:58:10.115 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: eph osd last-stat-seq 1 2026-02-17T05:58:10.115 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705702 2026-02-17T05:58:10.115 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.115 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542180 2026-02-17T05:58:10.115 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.115 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542180 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542180 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542180' 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.2 seq 60129542180 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542180 -lt 60129542180 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874258 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.116 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874258 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874258 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874258' 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.3 seq 279172874258 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874258 -lt 279172874258 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280545 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:10.117 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280545 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280545 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280545' 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.4 seq 94489280545 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280545 -lt 94489280545 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.118 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-382252089347 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.clceph osd last-stat-seq 1 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705701 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542179 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542179 2026-02-17T05:58:10.119 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542179 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542179' 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.2 seq 60129542179 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542180 -lt 60129542179 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874257 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874257 2026-02-17T05:58:10.120 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874257 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874257' 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.3 seq 279172874257 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874258 -lt 279172874257 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280544 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280544 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.121 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280544 2026-02-17T05:58:10.122 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280544' 2026-02-17T05:58:10.122 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.4 seq 94489280544 2026-02-17T05:58:10.122 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:10.122 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280545 -lt 94489280544 2026-02-17T05:58:10.122 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.122 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-382252089346 2026-02-17T05:58:10.592 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.cient.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.593 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:10.593 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-382252089347 2026-02-17T05:58:10.593 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.593 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089347 2026-02-17T05:58:10.593 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 382252089347' 2026-02-17T05:58:10.593 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.5 seq 382252089347 2026-02-17T05:58:10.593 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089347 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018909 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018909 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.594 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018909 2026-02-17T05:58:10.595 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018909' 2026-02-17T05:58:10.595 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.6 seq 128849018909 2026-02-17T05:58:10.595 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:10.595 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018909 -lt 128849018909 2026-02-17T05:58:10.595 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.595 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888090 2026-02-17T05:58:10.595 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888090 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888090 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888090' 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.7 seq 146028888090 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888091 -lt 146028888090 2026-02-17T05:58:10.596 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.597 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724569 2026-02-17T05:58:10.597 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.597 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:10.597 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724569 2026-02-17T05:58:10.597 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.598 INFO:tasks.workunit.client.0.trial074.stderr:23lient.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.598 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:10.598 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-382252089346 2026-02-17T05:58:10.598 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089346 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 382252089346' 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.5 seq 382252089346 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089346 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018908 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.599 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018908 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018908 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018908' 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.6 seq 128849018908 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018909 -lt 128849018908 2026-02-17T05:58:10.600 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888091 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888091 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888091 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888091' 2026-02-17T05:58:10.601 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.7 seq 146028888091 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888091 -lt 146028888091 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724570 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724570 2026-02-17T05:58:10.602 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:11.417 INFO:tasks.workunit.client.0.trial074.stderr:23284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724569 2026-02-17T05:58:11.417 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724569' 2026-02-17T05:58:11.417 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.8 seq 167503724569 2026-02-17T05:58:11.417 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724570 -lt 167503724569 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-382252089346 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-382252089346 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089346 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 382252089346' 2026-02-17T05:58:11.418 INFO:tasks.workunit.client.0.trial074.stderr:233284: waiting osd.9 seq 382252089346 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089346 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:11.419 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:233284: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:11.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:33282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724570 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724570' 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.8 seq 167503724570 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724570 -lt 167503724570 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-382252089347 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:11.431 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-382252089347 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089347 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 382252089347' 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: waiting osd.9 seq 382252089347 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089347 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:11.432 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:11.433 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:11.434 INFO:tasks.workunit.client.0.trial074.stderr:233282: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:11.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:11.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:11.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 233281 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:58:11.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:58:11.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:58:11.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:58:11.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:58:11.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:58:11.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:11.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:11.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:11.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:11.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:57:46.331719+0000 2026-02-17T05:58:11.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:58:11.882 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:58:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:57:46.331719+0000 2026-02-17T05:58:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:58:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:57:46.331719+0000 2026-02-17T05:58:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:58:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:58:11.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:11.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:11.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:11.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:11.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:11.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:12.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:57:46.331719+0000 '>' 2026-02-17T05:57:46.331719+0000 2026-02-17T05:58:12.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:58:13.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:58:13.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:13.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:13.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:13.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:13.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:13.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:13.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:57:46.331719+0000 '>' 2026-02-17T05:57:46.331719+0000 2026-02-17T05:58:13.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:58:14.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:58:14.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:14.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:14.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:14.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:14.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:14.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:58:12.330008+0000 '>' 2026-02-17T05:57:46.331719+0000 2026-02-17T05:58:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:58:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:58:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 236739"' 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 236739' 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 236740"' 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 236740' 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 236739 236740' 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:14.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 236739 2026-02-17T05:58:14.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:14.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/236742: /' 2026-02-17T05:58:14.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:58:14.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/236744: /' 2026-02-17T05:58:16.501 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.501 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:16.501 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:58:16.501 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:16.502 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: _ 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: hinfo_key 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: snapset 2026-02-17T05:58:16.503 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:58:16.504 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.504 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:16.504 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:58:16.504 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:16.504 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:16.504 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:16.504 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:16.505 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:16.505 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:16.505 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:58:16.505 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:58:16.505 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:16.506 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:58:16.507 INFO:tasks.workunit.client.0.trial074.stderr:236742: start osd.5 2026-02-17T05:58:16.521 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_rec236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.521 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:16.521 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T05:58:16.521 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:16.521 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:58:16.521 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.521 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T05:58:16.522 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: _ 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: hinfo_key 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: snapset 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:16.523 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:16.524 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T05:58:16.524 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:16.524 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T05:58:16.524 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:16.524 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:16.524 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:16.525 INFO:tasks.workunit.client.0.trial074.stderr:236744: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:16.526 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T05:58:16.527 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T05:58:16.527 INFO:tasks.workunit.client.0.trial074.stderr:236744: start osd.9 2026-02-17T05:58:20.819 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:58:20.819 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:58:20.819 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:58:20.819 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:20.819 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 2026-02-17T05:58:16.506+0000 7f88cc5ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 2026-02-17T05:58:16.510+0000 7f88cc5ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 2026-02-17T05:58:16.510+0000 7f88cc5ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 0 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 1 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 2026-02-17T05:58:17.802+0000 7f88cc5ab8c0 -1 Falling back to public interface 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 2 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: 3 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:20.820 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236742: 2026-02-17T05:58:20.806+0000 7f8overy_ops 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: 2026-02-17T05:58:16.526+0000 7f42daf0a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: 2026-02-17T05:58:16.526+0000 7f42daf0a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:20.848 INFO:tasks.workunit.client.0.trial074.stderr:236744: 2026-02-17T05:58:16.526+0000 7f42daf0a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: 0 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: 1 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: 2026-02-17T05:58:17.814+0000 7f42daf0a8c0 -1 Falling back to public interface 2026-02-17T05:58:20.849 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: 2 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: 3 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:20.850 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:21.763 INFO:tasks.workunit.client.0.trial074.stderr:236744: 2026-02-17T05:58:20.834+0000 7f42daf0a8c0 -1 osd.9 90 log_to_monitors true 2026-02-17T05:58:21.763 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:21.763 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:21.763 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: 4 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: osd.9 up in weight 1 up_from 95 up_thru 0 down_at 91 last_clean_interval [89,90) [v2:127.0.0.1:6874/2883322097,v1:127.0.0.1:6875/2883322097] [v2:127.0.0.1:6876/2883322097,v1:127.0.0.1:6877/2883322097] exists,up d332f160-e7d8-409f-b480-a33a4da05bef 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:21.764 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:21.765 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 1 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 2 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 3 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 4 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 5 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 6 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 7 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 8 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: 9' 2026-02-17T05:58:21.766 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:21.767 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:21.767 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:21.767 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836523 2026-02-17T05:58:21.767 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836523 2026-02-17T05:58:21.767 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523' 2026-02-17T05:58:21.767 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids8cc5ab8c0 -1 osd.5 90 log_to_monitors true 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: 4 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: osd.5 up in weight 1 up_from 95 up_thru 95 down_at 91 last_clean_interval [89,90) [v2:127.0.0.1:6842/4184455170,v1:127.0.0.1:6843/4184455170] [v2:127.0.0.1:6844/4184455170,v1:127.0.0.1:6845/4184455170] exists,up d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:21.768 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:21.769 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 1 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 2 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 3 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 4 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 5 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 6 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 7 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 8 2026-02-17T05:58:21.770 INFO:tasks.workunit.client.0.trial074.stderr:236742: 9' 2026-02-17T05:58:21.771 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:21.771 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:21.771 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:21.771 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836524 2026-02-17T05:58:21.771 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836524 2026-02-17T05:58:21.771 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524' 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $id 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705705 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705705 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705' 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:22.263 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542185 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542185 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185' 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874262 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874262 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185 3-279172874262' 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280548 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280548 2026-02-17T05:58:22.264 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185 3-279172874262 4-94489280548' 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893122 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893122 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185 3-279172874262 4-94489280548 5-408021893122' 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018913 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018913 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185 3-279172874262 4-94489280548 5-408021893122 6-128849018913' 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.265 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:58:22.266 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: fs 2026-02-17T05:58:22.266 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:22.266 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705706 2026-02-17T05:58:22.266 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705706 2026-02-17T05:58:22.266 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706' 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542184 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542184 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184' 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874261 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874261 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184 3-279172874261' 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.267 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280549 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280549 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184 3-279172874261 4-94489280549' 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893123 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893123 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184 3-279172874261 4-94489280549 5-408021893123' 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:22.268 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018914 2026-02-17T05:58:22.269 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018914 2026-02-17T05:58:22.269 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184 3-279172874261 4-94489280549 5-408021893123 6-128849018914' 2026-02-17T05:58:22.269 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:22.269 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:58:23.718 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888096 2026-02-17T05:58:23.718 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888096 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184 3-279172874261 4-94489280549 5-408021893123 6-128849018914 7-146028888096' 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724574 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724574 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184 3-279172874261 4-94489280549 5-408021893123 6-128849018914 7-146028888096 8-167503724574' 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893123 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893123 2026-02-17T05:58:23.719 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542184 3-279172874261 4-94489280549 5-408021893123 6-128849018914 7-146028888096 8-167503724574 9-408021893123' 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836524 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836524 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836524 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836524' 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.0 seq 21474836524 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:23.720 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836524 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836524 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705706 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:23.721 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705706 2026-02-17T05:58:23.722 INFO:tasks.workunit.client.0.trial074.stderr:236742: //holush_pg_stats: seq=146028888095 2026-02-17T05:58:23.722 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888095 2026-02-17T05:58:23.722 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185 3-279172874262 4-94489280548 5-408021893122 6-128849018913 7-146028888095' 2026-02-17T05:58:23.722 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:23.722 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:23.722 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724573 2026-02-17T05:58:23.722 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724573 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185 3-279172874262 4-94489280548 5-408021893122 6-128849018913 7-146028888095 8-167503724573' 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893122 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893122 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705705 2-60129542185 3-279172874262 4-94489280548 5-408021893122 6-128849018913 7-146028888095 8-167503724573 9-408021893122' 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836523 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836523 2026-02-17T05:58:23.723 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836523 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836523' 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.0 seq 21474836523 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836523 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836523 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705705 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:23.724 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:23.725 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705705 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236744: //homme/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705706 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705706' 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.1 seq 38654705706 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705707 -lt 38654705706 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542184 2026-02-17T05:58:24.187 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542184 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542184 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542184' 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.2 seq 60129542184 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542185 -lt 60129542184 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874261 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874261 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874261 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874261' 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.3 seq 279172874261 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874263 -lt 279172874261 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280549 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280549 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280549 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280549' 2026-02-17T05:58:24.188 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.4 seq 94489280549 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.ce/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705705 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705705' 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.1 seq 38654705705 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705707 -lt 38654705705 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542185 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542185 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542185 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542185' 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.2 seq 60129542185 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542185 -lt 60129542185 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874262 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874262 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874262 2026-02-17T05:58:24.192 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874262' 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.3 seq 279172874262 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874263 -lt 279172874262 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280548 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280548 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280548 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280548' 2026-02-17T05:58:24.193 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.4 seq 94489280548 2026-02-17T05:58:24.807 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.cllient.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:24.807 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280549 -lt 94489280549 2026-02-17T05:58:24.807 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-408021893123 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-408021893123 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893123 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 408021893123' 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.5 seq 408021893123 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.808 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018914 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018914 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018914 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018914' 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.6 seq 128849018914 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018914 -lt 128849018914 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888096 2026-02-17T05:58:24.809 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888096 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888096 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888096' 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.7 seq 146028888096 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888096 -lt 146028888096 2026-02-17T05:58:24.810 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: ient.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280549 -lt 94489280548 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-408021893122 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-408021893122 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893122 2026-02-17T05:58:24.816 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 408021893122' 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.5 seq 408021893122 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893123 -lt 408021893122 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018913 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018913 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018913 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018913' 2026-02-17T05:58:24.817 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.6 seq 128849018913 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018914 -lt 128849018913 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888095 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888095 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888095 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888095' 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.7 seq 146028888095 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:24.818 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888096 -lt 146028888095 2026-02-17T05:58:24.819 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:25.460 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: fflush_pg_stats: echo 8-167503724574 2026-02-17T05:58:25.460 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:25.460 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:25.460 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724574 2026-02-17T05:58:25.460 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:25.460 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724574 2026-02-17T05:58:25.460 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724574' 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.8 seq 167503724574 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724574 -lt 167503724574 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-408021893123 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-408021893123 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:25.461 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893123 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 408021893123' 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: waiting osd.9 seq 408021893123 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:25.462 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:25.463 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:25.463 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:25.463 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:25.463 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:25.463 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:25.463 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:25.463 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:25.469 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json slush_pg_stats: echo 8-167503724573 2026-02-17T05:58:25.469 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:25.469 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724573 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724573 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724573' 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.8 seq 167503724573 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724574 -lt 167503724573 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-408021893122 2026-02-17T05:58:25.470 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-408021893122 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893122 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 408021893122' 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: waiting osd.9 seq 408021893122 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893123 -lt 408021893122 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:25.471 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:25.472 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:25.637 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json sttatus 2026-02-17T05:58:25.638 INFO:tasks.workunit.client.0.trial074.stderr:236742: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:25.638 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:25.638 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:25.638 INFO:tasks.workunit.client.0.trial074.stderr:236742: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:25.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:25.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:25.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 236740 2026-02-17T05:58:25.657 INFO:tasks.workunit.client.0.trial074.stderr:atus 2026-02-17T05:58:25.657 INFO:tasks.workunit.client.0.trial074.stderr:236744: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:25.658 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:25.658 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:25.658 INFO:tasks.workunit.client.0.trial074.stderr:236744: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:58:25.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:58:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:58:25.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:58:25.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:58:25.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:58:25.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:58:25.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 5 2026-02-17T05:58:25.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T05:58:25.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T05:58:25.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T05:58:25.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=5 2026-02-17T05:58:25.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 239756"' 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 239756' 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 239757"' 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 239757' 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T05:58:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 239756 239757' 2026-02-17T05:58:25.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:58:25.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:25.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 239756 2026-02-17T05:58:25.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:58:25.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/239759: /' 2026-02-17T05:58:25.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:25.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/239761: /' 2026-02-17T05:58:28.171 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:58:28.172 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:28.173 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:28.174 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:28.175 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:28.175 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:28.175 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:58:28.175 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:28.176 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: start osd.5 2026-02-17T05:58:28.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:58:28.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T05:58:28.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:28.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:28.257 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:28.258 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:28.259 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:28.259 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:58:28.259 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:58:28.259 INFO:tasks.workunit.client.0.trial074.stderr:239759: start osd.3 2026-02-17T05:58:28.259 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:58:32.575 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: 2026-02-17T05:58:28.258+0000 7fc7087008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: 2026-02-17T05:58:28.258+0000 7fc7087008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: 2026-02-17T05:58:28.258+0000 7fc7087008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:58:32.576 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: 0 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: 2026-02-17T05:58:29.542+0000 7fc7087008c0 -1 Falling back to public interface 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:32.577 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: 1 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: 2 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:32.578 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:32.579 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:32.579 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.579 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:32.579 INFO:tasks.workunit.client.0.trial074.stderr:239759: 3 2026-02-17T05:58:32.579 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.579 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:32.579 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:32.756 INFO:tasks.workunit.client.0.trial074.stderr:239759: 2026-02-17T05:58:32.562+0000 7fc7087008c0 239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:58:32.756 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:58:32.756 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:32.756 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:32.756 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:32.756 INFO:tasks.workunit.client.0.trial074.stderr:239761: 2026-02-17T05:58:28.186+0000 7f0ad5f6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: 2026-02-17T05:58:28.186+0000 7f0ad5f6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: 2026-02-17T05:58:28.186+0000 7f0ad5f6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: 0 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.757 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: 1 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: 2026-02-17T05:58:29.694+0000 7f0ad5f6c8c0 -1 Falling back to public interface 2026-02-17T05:58:32.758 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: 2 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: 3 2026-02-17T05:58:32.759 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:32.760 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:32.760 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:33.425 INFO:tasks.workunit.client.0.trial074.stderr:239761: 2026-02-17T05:58:32.742+0000 7f0ad5f6c8c0 -1 osd.5 96 log_to_monitors true 2026-02-17T05:58:33.425 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:33.425 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: 4 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: osd.5 up in weight 1 up_from 103 up_thru 103 down_at 97 last_clean_interval [95,96) [v2:127.0.0.1:6842/3701950353,v1:127.0.0.1:6843/3701950353] [v2:127.0.0.1:6844/3701950353,v1:127.0.0.1:6845/3701950353] exists,up d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:33.426 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:33.427 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 1 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 2 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 3 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 4 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 5 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 6 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 7 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 8 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: 9' 2026-02-17T05:58:33.428 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:33.429 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.429 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:33.429 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836527 2026-02-17T05:58:33.429 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836527 2026-02-17T05:58:33.429 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527' 2026-02-17T05:58:33.429 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.514 INFO:tasks.workunit.client.0.trial074.stderr:239761-1 osd.3 96 log_to_monitors true 2026-02-17T05:58:33.514 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:33.514 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:33.514 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: 4 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: osd.3 up in weight 1 up_from 101 up_thru 104 down_at 97 last_clean_interval [65,96) [v2:127.0.0.1:6826/3074451033,v1:127.0.0.1:6827/3074451033] [v2:127.0.0.1:6828/3074451033,v1:127.0.0.1:6829/3074451033] exists,up be445b3b-0d24-49e2-bbc9-ac29a33acfe3 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:33.515 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:33.516 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 1 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 2 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 3 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 4 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 5 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 6 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 7 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 8 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: 9' 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.517 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:33.518 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836528 2026-02-17T05:58:33.518 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836528 2026-02-17T05:58:33.518 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528' 2026-02-17T05:58:33.518 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705710 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705710 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710' 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542188 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542188 2026-02-17T05:58:33.942 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188' 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696898 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696898 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188 3-433791696898' 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280552 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280552 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188 3-433791696898 4-94489280552' 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=442381631490 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 442381631490 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188 3-433791696898 4-94489280552 5-442381631490' 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018917 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018917 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188 3-433791696898 4-94489280552 5-442381631490 6-128849018917' 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:33.943 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:58:34.011 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:34.011 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705711 2026-02-17T05:58:34.011 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705711 2026-02-17T05:58:34.011 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711' 2026-02-17T05:58:34.011 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542189 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542189 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189' 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696899 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696899 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-433791696899' 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:34.012 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280554 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280554 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-433791696899 4-94489280554' 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=442381631491 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 442381631491 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-433791696899 4-94489280554 5-442381631491' 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018918 2026-02-17T05:58:34.013 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018918 2026-02-17T05:58:34.014 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-433791696899 4-94489280554 5-442381631491 6-128849018918' 2026-02-17T05:58:34.014 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:34.014 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:58:35.473 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888100 2026-02-17T05:58:35.473 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888100 2026-02-17T05:58:35.473 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-433791696899 4-94489280554 5-442381631491 6-128849018918 7-146028888100' 2026-02-17T05:58:35.473 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:35.473 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:35.473 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724579 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724579 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-433791696899 4-94489280554 5-442381631491 6-128849018918 7-146028888100 8-167503724579' 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893127 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893127 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-433791696899 4-94489280554 5-442381631491 6-128849018918 7-146028888100 8-167503724579 9-408021893127' 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836528 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:35.474 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836528 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836528 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836528' 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.0 seq 21474836528 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836528 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836529 -lt 21474836528 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:35.475 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705711 2026-02-17T05:58:35.476 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:35.476 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:35.476 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705711 2026-02-17T05:58:35.972 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:35.972 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705711 2026-02-17T05:58:35.972 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705711' 2026-02-17T05:58:35.972 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.1 seq 38654705711 2026-02-17T05:58:35.972 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:58:35.972 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705711 -lt 38654705711 2026-02-17T05:58:35.972 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542189 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542189 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542189 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542189' 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.2 seq 60129542189 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542189 -lt 60129542189 2026-02-17T05:58:35.973 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-433791696899 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-433791696899 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696899 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 433791696899' 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.3 seq 433791696899 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696899 -lt 433791696899 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:35.974 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280554 2026-02-17T05:58:35.975 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:35.975 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:35.975 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280554 2026-02-17T05:58:35.975 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:35.975 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280554 2026-02-17T05:58:35.975 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280554' 2026-02-17T05:58:35.975 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.4 seq 94489280554 2026-02-17T05:58:36.551 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/_stats: seq=146028888099 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888099 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188 3-433791696898 4-94489280552 5-442381631490 6-128849018917 7-146028888099' 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724578 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724578 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188 3-433791696898 4-94489280552 5-442381631490 6-128849018917 7-146028888099 8-167503724578' 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:36.552 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893126 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893126 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705710 2-60129542188 3-433791696898 4-94489280552 5-442381631490 6-128849018917 7-146028888099 8-167503724578 9-408021893126' 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836527 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836527 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836527 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836527' 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.0 seq 21474836527 2026-02-17T05:58:36.553 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836527 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836527 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:36.554 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836529 -lt 21474836527 2026-02-17T05:58:36.607 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $sqa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:36.607 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280554 -lt 94489280554 2026-02-17T05:58:36.607 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-442381631491 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-442381631491 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=442381631491 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 442381631491' 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.5 seq 442381631491 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 442381631491 -lt 442381631491 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018918 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018918 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018918 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018918' 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.6 seq 128849018918 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018918 -lt 128849018918 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888100 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888100 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888100 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888100' 2026-02-17T05:58:36.608 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.7 seq 146028888100 2026-02-17T05:58:36.609 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:36.609 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888101 -lt 146028888100 2026-02-17T05:58:36.609 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pgeqs 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705710 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705710 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705710 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705710' 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.1 seq 38654705710 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705711 -lt 38654705710 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542188 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542188 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542188 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542188' 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.2 seq 60129542188 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:37.046 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542189 -lt 60129542188 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-433791696898 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-433791696898 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696898 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 433791696898' 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.3 seq 433791696898 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696899 -lt 433791696898 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280552 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.047 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:37.253 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_stats: echo 8-167503724579 2026-02-17T05:58:37.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:37.253 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724579 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724579 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724579' 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.8 seq 167503724579 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724579 -lt 167503724579 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-408021893127 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-408021893127 2026-02-17T05:58:37.254 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893127 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 408021893127' 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: waiting osd.9 seq 408021893127 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893127 -lt 408021893127 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:37.255 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:37.256 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:37.256 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:37.256 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:37.256 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:37.256 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:37.256 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:37.431 INFO:tasks.workunit.client.0.trial074.stderr:239759: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:37.432 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:37.432 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:37.432 INFO:tasks.workunit.client.0.trial074.stderr:239759: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:37.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:37.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:37.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 239757 2026-02-17T05:58:37.529 INFO:tasks.workunit.client.0.trial074.stderr:_pg_stats: echo 4-94489280552 2026-02-17T05:58:37.529 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.529 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280552 2026-02-17T05:58:37.529 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280552' 2026-02-17T05:58:37.529 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.4 seq 94489280552 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280554 -lt 94489280552 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-442381631490 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-442381631490 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=442381631490 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 442381631490' 2026-02-17T05:58:37.530 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.5 seq 442381631490 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 442381631491 -lt 442381631490 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018917 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018917 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018917 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018917' 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.6 seq 128849018917 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:37.531 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018919 -lt 128849018917 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888099 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888099 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888099 2026-02-17T05:58:37.532 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888099' 2026-02-17T05:58:38.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.7 seq 146028888099 2026-02-17T05:58:38.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:38.177 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888101 -lt 146028888099 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724578 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724578 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724578 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724578' 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.8 seq 167503724578 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724579 -lt 167503724578 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-408021893126 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-408021893126 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:38.178 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893126 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 408021893126' 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: waiting osd.9 seq 408021893126 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893128 -lt 408021893126 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:38.179 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:38.511 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:38.511 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:38.512 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:38.512 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:38.512 INFO:tasks.workunit.client.0.trial074.stderr:239761: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:38.512 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:38.512 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:38.512 INFO:tasks.workunit.client.0.trial074.stderr:239761: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:38.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:38.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:58:38.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:58:38.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:58:38.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T05:58:38.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:58:38.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T05:58:38.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T05:58:38.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:58:38.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T05:58:38.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:58:38.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T05:58:38.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T05:58:38.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T05:58:38.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T05:58:38.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:38.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:38.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:38.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:38.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:38.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T05:58:38.958 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T05:58:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T05:58:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:58:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:58:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:38.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:38.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:38.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:38.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:38.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:39.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:58:12.330008+0000 '>' 2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:39.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:58:40.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:58:40.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:40.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:40.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:40.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:40.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:40.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:40.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:58:12.330008+0000 '>' 2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:40.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:58:41.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:58:41.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:41.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:41.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:41.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:41.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:41.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:41.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:58:12.330008+0000 '>' 2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:41.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:58:42.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:58:42.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:42.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:42.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:42.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:42.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:42.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:42.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:58:12.330008+0000 '>' 2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:42.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:58:43.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:58:43.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:58:43.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T05:58:43.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T05:58:43.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:58:43.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:58:43.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:58:39.654713+0000 '>' 2026-02-17T05:58:12.330008+0000 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 243336"' 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 243336' 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 243337"' 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 243337' 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 243336 243337' 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:43.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 243336 2026-02-17T05:58:43.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:58:43.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/243339: /' 2026-02-17T05:58:43.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:43.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/243341: /' 2026-02-17T05:58:45.913 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.913 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:45.913 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T05:58:45.913 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:45.913 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:58:45.913 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:45.914 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: _ 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: hinfo_key 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: snapset 2026-02-17T05:58:45.915 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T05:58:45.916 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.916 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:45.916 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T05:58:45.916 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:45.916 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T05:58:45.916 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:45.917 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:45.918 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T05:58:45.919 INFO:tasks.workunit.client.0.trial074.stderr:243339: start osd.3 2026-02-17T05:58:45.933 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.933 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:58:45.933 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T05:58:45.933 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:58:45.933 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:45.934 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: _ 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: hinfo_key 2026-02-17T05:58:45.935 INFO:tasks.workunit.client.0.trial074.stderr:243341: snapset 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:45.936 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:58:45.937 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:45.938 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T05:58:45.939 INFO:tasks.workunit.client.0.trial074.stderr:243341: start osd.5 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reclen=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243339: 2026-02-17T05:58:45.922+0000 7fa0f73a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:49.692 INFO:tasks.workunit.client.0.trial074.stderr:243339: 2026-02-17T05:58:45.922+0000 7fa0f73a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: 2026-02-17T05:58:45.922+0000 7fa0f73a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: 0 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: 2026-02-17T05:58:47.190+0000 7fa0f73a48c0 -1 Falling back to public interface 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: 1 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: 2 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: 3 2026-02-17T05:58:49.693 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:49.694 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:50.298 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: 2026-02-17T05:58:45.938+0000 7fc89993f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: 2026-02-17T05:58:45.938+0000 7fc89993f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: 2026-02-17T05:58:45.942+0000 7fc89993f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T05:58:50.299 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: 0 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: 2026-02-17T05:58:47.226+0000 7fc89993f8c0 -1 Falling back to public interface 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:50.300 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: 1 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: 2 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:50.301 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: 3 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:50.302 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:51.172 INFO:tasks.workunit.client.0.trial074.stderr:243341: 2026-02-17T05:58:50.286+0000 7fc89993f8c0 -1 osd.5 105 log_to_monitors true 2026-02-17T05:58:51.172 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:51.172 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: 4 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: osd.5 up in weight 1 up_from 110 up_thru 110 down_at 106 last_clean_interval [103,105) [v2:127.0.0.1:6842/2263246391,v1:127.0.0.1:6843/2263246391] [v2:127.0.0.1:6844/2263246391,v1:127.0.0.1:6845/2263246391] exists,up d39da59b-0cad-49f9-b89a-4a273b142b64 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:51.173 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:51.174 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 1 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 2 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 3 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 4 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 5 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 6 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 7 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 8 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: 9' 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:51.175 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.176 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:51.176 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836533 2026-02-17T05:58:51.176 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836533 2026-02-17T05:58:51.176 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533' 2026-02-17T05:58:51.176 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd .0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: 2026-02-17T05:58:50.238+0000 7fa0f73a48c0 -1 osd.3 105 log_to_monitors true 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: 4 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: osd.3 up in weight 1 up_from 110 up_thru 110 down_at 106 last_clean_interval [101,105) [v2:127.0.0.1:6826/408209063,v1:127.0.0.1:6827/408209063] [v2:127.0.0.1:6828/408209063,v1:127.0.0.1:6829/408209063] exists,up be445b3b-0d24-49e2-bbc9-ac29a33acfe3 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:58:51.177 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:58:51.178 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 1 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 2 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 3 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 4 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 5 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 6 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 7 2026-02-17T05:58:51.179 INFO:tasks.workunit.client.0.trial074.stderr:243339: 8 2026-02-17T05:58:51.180 INFO:tasks.workunit.client.0.trial074.stderr:243339: 9' 2026-02-17T05:58:51.180 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:58:51.180 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.180 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:58:51.180 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836534 2026-02-17T05:58:51.180 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836534 2026-02-17T05:58:51.180 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534' 2026-02-17T05:58:51.605 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.605 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:51.605 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705716 2026-02-17T05:58:51.605 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705716 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716' 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542194 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542194 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194' 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402563 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402563 2026-02-17T05:58:51.606 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194 3-472446402563' 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280559 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280559 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194 3-472446402563 4-94489280559' 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402563 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402563 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194 3-472446402563 4-94489280559 5-472446402563' 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.607 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:51.608 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018924 2026-02-17T05:58:51.608 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018924 2026-02-17T05:58:51.608 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194 3-472446402563 4-94489280559 5-472446402563 6-128849018924' 2026-02-17T05:58:51.608 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.671 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tellin $ids 2026-02-17T05:58:51.671 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:58:51.671 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705715 2026-02-17T05:58:51.671 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705715 2026-02-17T05:58:51.671 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715' 2026-02-17T05:58:51.671 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542193 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542193 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193' 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402562 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402562 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193 3-472446402562' 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280558 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280558 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193 3-472446402562 4-94489280558' 2026-02-17T05:58:51.672 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402562 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402562 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193 3-472446402562 4-94489280558 5-472446402562' 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018923 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018923 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193 3-472446402562 4-94489280558 5-472446402562 6-128849018923' 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.673 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T05:58:51.979 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888105 2026-02-17T05:58:51.979 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888105 2026-02-17T05:58:51.979 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193 3-472446402562 4-94489280558 5-472446402562 6-128849018923 7-146028888105' 2026-02-17T05:58:51.979 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.979 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:51.979 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724583 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724583 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193 3-472446402562 4-94489280558 5-472446402562 6-128849018923 7-146028888105 8-167503724583' 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893132 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893132 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705715 2-60129542193 3-472446402562 4-94489280558 5-472446402562 6-128849018923 7-146028888105 8-167503724583 9-408021893132' 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836533 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836533 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836533 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836533' 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.0 seq 21474836533 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836533 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705715 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705715 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705715 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705715' 2026-02-17T05:58:51.980 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.1 seq 38654705715 2026-02-17T05:58:51.983 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: f osd.7 flush_pg_stats 2026-02-17T05:58:51.983 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888106 2026-02-17T05:58:51.983 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888106 2026-02-17T05:58:51.983 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194 3-472446402563 4-94489280559 5-472446402563 6-128849018924 7-146028888106' 2026-02-17T05:58:51.983 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.983 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T05:58:51.983 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724584 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724584 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194 3-472446402563 4-94489280559 5-472446402563 6-128849018924 7-146028888106 8-167503724584' 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893133 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893133 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705716 2-60129542194 3-472446402563 4-94489280559 5-472446402563 6-128849018924 7-146028888106 8-167503724584 9-408021893133' 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836534 2026-02-17T05:58:51.984 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836534 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836534 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836534' 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.0 seq 21474836534 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836534 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705716 2026-02-17T05:58:51.985 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:51.986 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:58:51.986 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705716 2026-02-17T05:58:51.986 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:51.986 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705716 2026-02-17T05:58:51.986 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705716' 2026-02-17T05:58:53.622 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.1 seq 38654705716 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705716 -lt 38654705716 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542194 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542194 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542194 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542194' 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.2 seq 60129542194 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542195 -lt 60129542194 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-472446402563 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-472446402563 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402563 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 472446402563' 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.3 seq 472446402563 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402563 -lt 472446402563 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280559 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280559 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280559 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280559' 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.4 seq 94489280559 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:53.623 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280557 -lt 94489280559 2026-02-17T05:58:53.624 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/stalush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705716 -lt 38654705715 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542193 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542193 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542193 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542193' 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.2 seq 60129542193 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542195 -lt 60129542193 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-472446402562 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-472446402562 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402562 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 472446402562' 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.3 seq 472446402562 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402563 -lt 472446402562 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-94489280558 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:53.631 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-94489280558 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280558 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 94489280558' 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.4 seq 94489280558 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280557 -lt 94489280558 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:58:53.632 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280559 -lt 94489280558 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-472446402562 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-472446402562 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402562 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 472446402562' 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.5 seq 472446402562 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402563 -lt 472446402562 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018923 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018923 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018923 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018923' 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.6 seq 128849018923 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018924 -lt 128849018923 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888105 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888105 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888105 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888105' 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.7 seq 146028888105 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888106 -lt 146028888105 2026-02-17T05:58:54.247 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:ndalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280559 -lt 94489280559 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-472446402563 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-472446402563 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402563 2026-02-17T05:58:54.268 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 472446402563' 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.5 seq 472446402563 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402563 -lt 472446402563 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-128849018924 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-128849018924 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018924 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 128849018924' 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.6 seq 128849018924 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018924 -lt 128849018924 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-146028888106 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.269 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T05:58:54.270 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-146028888106 2026-02-17T05:58:54.270 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.270 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888106 2026-02-17T05:58:54.270 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 146028888106' 2026-02-17T05:58:54.270 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.7 seq 146028888106 2026-02-17T05:58:54.270 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T05:58:54.270 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888106 -lt 146028888106 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_st2274: flush_pg_stats: echo 8-167503724583 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724583 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724583 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724583' 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.8 seq 167503724583 2026-02-17T05:58:54.890 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724584 -lt 167503724583 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-408021893132 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-408021893132 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893132 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 408021893132' 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: waiting osd.9 seq 408021893132 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893133 -lt 408021893132 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:54.891 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:54.925 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format ats: for s in $seqs 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724584 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724584 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724584 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724584' 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.8 seq 167503724584 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724584 -lt 167503724584 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-408021893133 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-408021893133 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893133 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 408021893133' 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: waiting osd.9 seq 408021893133 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893133 -lt 408021893133 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T05:58:54.926 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:58:55.072 INFO:tasks.workunit.client.0.trial074.stderr:243339: //homjson status 2026-02-17T05:58:55.072 INFO:tasks.workunit.client.0.trial074.stderr:243341: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:55.073 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:55.073 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:55.073 INFO:tasks.workunit.client.0.trial074.stderr:243341: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:55.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:55.112 INFO:tasks.workunit.client.0.trial074.stderr:e/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:58:55.113 INFO:tasks.workunit.client.0.trial074.stderr:243339: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:58:55.113 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T05:58:55.113 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:58:55.113 INFO:tasks.workunit.client.0.trial074.stderr:243339: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 243337 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T05:58:55.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:58:55.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:58:55.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T05:58:55.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:58:55.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:58:55.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:58:55.136 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:55.136 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:55.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:55.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:55.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:55.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:55.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:58:55.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:58:55.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:58:55.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:58:55.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:58:55.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:58:55.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:58:55.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:58:55.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:58:55.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:58:55.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:58:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:58:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:58:55.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:58:55.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:58:55.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:58:55.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:58:55.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:58:55.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:58:55.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:58:55.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:58:55.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T05:58:55.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T05:58:55.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T05:58:55.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T05:58:55.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T05:58:55.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T05:58:55.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:58:55.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T05:58:55.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T05:58:55.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T05:58:55.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T05:58:55.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T05:58:55.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T05:58:55.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T05:58:55.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T05:58:55.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T05:58:55.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T05:58:55.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T05:58:55.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T05:58:55.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T05:58:55.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T05:58:55.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T05:58:55.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T05:58:55.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T05:58:55.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_replicated td/osd-scrub-repair 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:94: TEST_corrupt_and_repair_replicated: local dir=td/osd-scrub-repair 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:95: TEST_corrupt_and_repair_replicated: local poolname=rbd 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:97: TEST_corrupt_and_repair_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T05:58:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T05:58:55.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T05:58:55.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:55.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:55.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:55.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:55.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T05:58:55.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T05:58:55.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T05:58:55.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:58:55.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:58:55.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T05:58:55.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T05:58:55.576 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:58:55.576 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:58:55.576 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:58:55.576 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:58:55.576 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.577 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.577 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:58:55.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:58:55.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T05:58:55.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T05:58:55.631 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.632 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.632 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T05:58:55.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T05:58:55.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T05:58:55.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:98: TEST_corrupt_and_repair_replicated: run_mgr td/osd-scrub-repair x 2026-02-17T05:58:55.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T05:58:55.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T05:58:55.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T05:58:55.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T05:58:55.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T05:58:55.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T05:58:55.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T05:58:55.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:55.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:55.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:55.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.783 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:55.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:58:55.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:99: TEST_corrupt_and_repair_replicated: run_osd td/osd-scrub-repair 0 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:58:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:58:55.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:58:55.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:58:55.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=88187236-4d09-4d15-9980-ef54bae7fbc6 2026-02-17T05:58:55.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 88187236-4d09-4d15-9980-ef54bae7fbc6' 2026-02-17T05:58:55.810 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 88187236-4d09-4d15-9980-ef54bae7fbc6 2026-02-17T05:58:55.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:58:55.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCfA5RpVyswMBAAcer9yC2NeURiW16uctktiw== 2026-02-17T05:58:55.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCfA5RpVyswMBAAcer9yC2NeURiW16uctktiw=="}' 2026-02-17T05:58:55.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 88187236-4d09-4d15-9980-ef54bae7fbc6 -i td/osd-scrub-repair/0/new.json 2026-02-17T05:58:55.907 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:58:55.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T05:58:55.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCfA5RpVyswMBAAcer9yC2NeURiW16uctktiw== --osd-uuid 88187236-4d09-4d15-9980-ef54bae7fbc6 2026-02-17T05:58:55.932 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:55.922+0000 7f653019b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:55.934 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:55.922+0000 7f653019b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:55.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:55.922+0000 7f653019b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:55.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:55.922+0000 7f653019b8c0 -1 bdev(0x5650d84afc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T05:58:55.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:55.922+0000 7f653019b8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T05:58:58.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T05:58:58.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:58:58.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T05:58:58.535 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T05:58:58.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:58:58.724 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T05:58:58.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T05:58:58.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:58:58.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:58:58.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:58:58.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:58:58.749 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:58.738+0000 7f8884ecb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:58.750 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:58.738+0000 7f8884ecb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:58.751 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:58:58.738+0000 7f8884ecb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:58:58.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T05:58:58.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:58:58.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:58:58.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:58:58.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:58:58.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:58:58.879 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:58:58.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:58:58.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:58:58.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:58:59.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:59:00.031 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:59:00.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:00.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:00.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:00.234 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:00.222+0000 7f8884ecb8c0 -1 Falling back to public interface 2026-02-17T05:59:01.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:01.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:01.184 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:59:01.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:59:01.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:01.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:01.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:02.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:02.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:02.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:59:02.336 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:59:02.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:02.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:02.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:03.222 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:03.210+0000 7f8884ecb8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T05:59:03.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:03.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:03.489 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:59:03.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:59:03.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:03.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:03.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:04.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:04.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:04.645 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:59:04.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:59:04.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:04.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:04.794 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1987247714,v1:127.0.0.1:6803/1987247714] [v2:127.0.0.1:6804/1987247714,v1:127.0.0.1:6805/1987247714] exists,up 88187236-4d09-4d15-9980-ef54bae7fbc6 2026-02-17T05:59:04.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:59:04.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:59:04.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:100: TEST_corrupt_and_repair_replicated: run_osd td/osd-scrub-repair 1 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:59:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:04.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:59:04.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T05:59:04.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=362b2a8d-5ec3-419d-8083-e6319212c2a2 2026-02-17T05:59:04.802 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 362b2a8d-5ec3-419d-8083-e6319212c2a2 2026-02-17T05:59:04.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 362b2a8d-5ec3-419d-8083-e6319212c2a2' 2026-02-17T05:59:04.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T05:59:04.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCoA5RpgILeLxAA9GF4uBpi8Fo9ijHj8cGcIA== 2026-02-17T05:59:04.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCoA5RpgILeLxAA9GF4uBpi8Fo9ijHj8cGcIA=="}' 2026-02-17T05:59:04.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 362b2a8d-5ec3-419d-8083-e6319212c2a2 -i td/osd-scrub-repair/1/new.json 2026-02-17T05:59:04.965 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:59:04.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T05:59:04.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCoA5RpgILeLxAA9GF4uBpi8Fo9ijHj8cGcIA== --osd-uuid 362b2a8d-5ec3-419d-8083-e6319212c2a2 2026-02-17T05:59:04.990 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:04.978+0000 7fc2608708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:04.991 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:04.978+0000 7fc2608708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:04.992 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:04.978+0000 7fc2608708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:04.993 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:04.982+0000 7fc2608708c0 -1 bdev(0x55e90df63c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T05:59:04.993 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:04.982+0000 7fc2608708c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T05:59:07.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T05:59:07.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T05:59:07.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T05:59:07.275 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T05:59:07.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T05:59:07.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T05:59:07.468 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T05:59:07.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:59:07.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T05:59:07.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T05:59:07.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T05:59:07.482 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:07.470+0000 7fad611478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:07.484 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:07.470+0000 7fad611478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:07.485 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:07.474+0000 7fad611478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:07.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:07.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:07.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:08.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:08.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:08.773 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T05:59:08.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:59:08.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:08.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:08.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:08.974 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:08.962+0000 7fad611478c0 -1 Falling back to public interface 2026-02-17T05:59:09.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:09.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:09.924 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T05:59:09.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:59:09.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:09.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:10.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:11.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:11.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:11.076 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T05:59:11.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:59:11.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:11.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:12.021 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:12.010+0000 7fad611478c0 -1 osd.1 0 log_to_monitors true 2026-02-17T05:59:12.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:12.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:12.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:59:12.226 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T05:59:12.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:12.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:12.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:13.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:13.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:13.394 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T05:59:13.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T05:59:13.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:13.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:13.546 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2957355261,v1:127.0.0.1:6811/2957355261] [v2:127.0.0.1:6812/2957355261,v1:127.0.0.1:6813/2957355261] exists,up 362b2a8d-5ec3-419d-8083-e6319212c2a2 2026-02-17T05:59:13.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:59:13.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:59:13.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:59:13.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:101: TEST_corrupt_and_repair_replicated: create_rbd_pool 2026-02-17T05:59:13.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T05:59:13.688 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T05:59:13.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T05:59:13.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T05:59:13.905 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T05:59:13.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T05:59:14.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T05:59:15.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:102: TEST_corrupt_and_repair_replicated: wait_for_clean 2026-02-17T05:59:15.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:59:15.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:59:15.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:59:15.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:59:15.169 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:59:15.169 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:59:15.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:59:15.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:59:15.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:59:15.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:59:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:59:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:59:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:59:15.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:59:15.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:59:15.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:59:15.437 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T05:59:15.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:59:15.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:15.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:59:15.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T05:59:15.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T05:59:15.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T05:59:15.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:15.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:59:15.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T05:59:15.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T05:59:15.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T05:59:15.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:15.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T05:59:15.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:15.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:59:15.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T05:59:15.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:15.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T05:59:15.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T05:59:15.583 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T05:59:15.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:15.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T05:59:15.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:59:16.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:59:16.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:16.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T05:59:16.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:16.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T05:59:16.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:16.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:59:16.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T05:59:16.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:16.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T05:59:16.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T05:59:16.890 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705666 2026-02-17T05:59:16.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:59:17.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T05:59:17.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:59:17.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:17.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:17.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:59:17.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:59:17.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:59:17.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:59:17.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:59:17.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:59:17.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:59:17.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:59:17.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:59:17.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:59:17.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:17.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:17.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:104: TEST_corrupt_and_repair_replicated: add_something td/osd-scrub-repair rbd 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T05:59:17.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T05:59:17.769 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T05:59:17.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T05:59:17.973 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T05:59:17.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T05:59:17.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T05:59:17.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T05:59:18.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:105: TEST_corrupt_and_repair_replicated: get_not_primary rbd SOMETHING 2026-02-17T05:59:18.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=rbd 2026-02-17T05:59:18.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T05:59:18.011 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary rbd SOMETHING 2026-02-17T05:59:18.011 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T05:59:18.012 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:59:18.012 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T05:59:18.012 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:59:18.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T05:59:18.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T05:59:18.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T05:59:18.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:105: TEST_corrupt_and_repair_replicated: corrupt_and_repair_one td/osd-scrub-repair rbd 0 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=rbd 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=0 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T05:59:18.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:59:18.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:59:19.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T05:59:20.274 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:59:20.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:59:20.812 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:59:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:59:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:59:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T05:59:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T05:59:20.814 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T05:59:20.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:59:20.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T05:59:20.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T05:59:20.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:59:20.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:59:20.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:59:20.838 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:20.825+0000 7fb8fe51c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:20.839 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:20.825+0000 7fb8fe51c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:20.840 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:20.825+0000 7fb8fe51c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:20.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T05:59:20.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:59:20.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T05:59:20.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:59:20.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:59:20.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:20.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:59:20.967 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:59:20.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:20.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:21.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:22.098 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:22.085+0000 7fb8fe51c8c0 -1 Falling back to public interface 2026-02-17T05:59:22.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:22.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:22.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:59:22.122 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:59:22.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:22.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:22.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:23.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:23.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:23.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:59:23.271 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:59:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:23.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:24.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:24.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:24.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:59:24.422 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:59:24.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:24.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:24.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:25.060 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:25.045+0000 7fb8fe51c8c0 -1 osd.0 17 log_to_monitors true 2026-02-17T05:59:25.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:25.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:25.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:59:25.574 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:59:25.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:25.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T05:59:25.725 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 22 up_thru 22 down_at 18 last_clean_interval [5,17) [v2:127.0.0.1:6802/3984004902,v1:127.0.0.1:6803/3984004902] [v2:127.0.0.1:6804/3984004902,v1:127.0.0.1:6805/3984004902] exists,up 88187236-4d09-4d15-9980-ef54bae7fbc6 2026-02-17T05:59:25.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:59:25.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:59:25.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:59:25.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:59:25.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:59:25.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:59:25.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:59:25.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:59:25.727 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:59:25.727 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:59:25.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:59:25.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:59:25.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:59:25.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:59:25.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:59:25.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:59:25.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:59:25.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:59:25.828 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:59:25.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:59:25.972 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T05:59:25.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:59:25.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:25.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:59:26.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280514 2026-02-17T05:59:26.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280514 2026-02-17T05:59:26.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280514' 2026-02-17T05:59:26.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:26.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:59:26.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T05:59:26.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T05:59:26.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280514 1-38654705669' 2026-02-17T05:59:26.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:26.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-94489280514 2026-02-17T05:59:26.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:26.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:59:26.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-94489280514 2026-02-17T05:59:26.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:26.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280514 2026-02-17T05:59:26.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 94489280514' 2026-02-17T05:59:26.116 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 94489280514 2026-02-17T05:59:26.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:26.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 94489280514 2026-02-17T05:59:26.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:59:27.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:59:27.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:27.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 94489280514 2026-02-17T05:59:27.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:59:28.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T05:59:28.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:28.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280514 -lt 94489280514 2026-02-17T05:59:28.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:28.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T05:59:28.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:28.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:59:28.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T05:59:28.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:28.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T05:59:28.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T05:59:28.580 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705669 2026-02-17T05:59:28.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:59:28.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T05:59:28.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:59:28.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:28.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:59:28.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:59:29.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:59:29.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:59:29.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:29.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:29.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:59:29.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:59:29.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:59:29.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-17T05:59:29.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T05:59:29.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:59:29.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T05:59:29.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:59:29.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=1.3 2026-02-17T05:59:29.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 1.3 2026-02-17T05:59:29.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.3 2026-02-17T05:59:29.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.3 2026-02-17T05:59:29.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:29.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:29.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:29.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:29.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:29.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.3 2026-02-17T05:59:29.710 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-17T05:59:29.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.3 2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:29.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T05:59:29.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:29.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:59:29.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:59:29.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:29.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:29.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:29.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:29.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:29.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:29.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:13.896058+0000 '>' 2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:29.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:59:30.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:59:30.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:30.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:30.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:30.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:30.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:30.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:31.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:13.896058+0000 '>' 2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:31.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:59:32.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:59:32.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:32.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:32.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:32.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:32.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:32.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:32.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:13.896058+0000 '>' 2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:32.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:59:33.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:59:33.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:33.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:33.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:33.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:33.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:33.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:33.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:13.896058+0000 '>' 2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:33.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:59:34.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:59:34.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:34.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:34.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:34.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:34.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:34.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:34.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:30.018762+0000 '>' 2026-02-17T05:59:13.896058+0000 2026-02-17T05:59:34.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:59:34.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:59:34.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:59:34.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:59:34.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:59:34.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:59:34.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:59:34.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:59:34.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:59:34.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:59:34.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:59:34.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:59:34.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:59:34.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:59:34.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:59:34.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:59:34.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:59:35.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:59:35.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:59:35.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:59:35.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:59:35.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:59:35.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:59:35.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:59:35.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:59:36.120 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T05:59:36.121 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:59:36.121 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:59:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:59:36.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:59:36.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:59:36.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:59:36.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:59:36.655 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:59:36.655 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:59:36.655 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:59:36.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:59:36.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:59:36.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:59:36.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:59:36.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:59:36.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:59:36.657 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:59:36.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:59:36.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:59:36.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:59:36.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:59:36.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:59:36.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:59:36.676 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:36.661+0000 7f1913ca88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:36.677 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:36.665+0000 7f1913ca88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:36.678 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:36.665+0000 7f1913ca88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:59:36.810 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:59:36.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:36.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:36.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:37.915 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:37.901+0000 7f1913ca88c0 -1 Falling back to public interface 2026-02-17T05:59:37.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:37.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:37.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:59:37.964 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:59:37.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:37.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:38.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:39.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:39.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:39.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:59:39.116 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:59:39.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:39.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:39.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:40.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:40.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:40.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:59:40.269 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:59:40.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:40.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:40.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:40.938 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:40.925+0000 7f1913ca88c0 -1 osd.1 25 log_to_monitors true 2026-02-17T05:59:41.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:41.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:41.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:59:41.421 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:59:41.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:41.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:41.573 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 26 last_clean_interval [9,25) [v2:127.0.0.1:6810/1302333811,v1:127.0.0.1:6811/1302333811] [v2:127.0.0.1:6812/1302333811,v1:127.0.0.1:6813/1302333811] exists,up 362b2a8d-5ec3-419d-8083-e6319212c2a2 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:59:41.574 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:59:41.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:59:41.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:59:41.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:59:41.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:59:41.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:59:41.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:59:41.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:59:41.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:59:41.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:59:41.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:59:41.839 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T05:59:41.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:59:41.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:41.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:59:41.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280518 2026-02-17T05:59:41.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280518 2026-02-17T05:59:41.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280518' 2026-02-17T05:59:41.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:41.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:59:41.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T05:59:41.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T05:59:41.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280518 1-124554051586' 2026-02-17T05:59:41.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:41.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-94489280518 2026-02-17T05:59:41.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:41.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:59:41.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-94489280518 2026-02-17T05:59:41.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:41.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280518 2026-02-17T05:59:41.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 94489280518' 2026-02-17T05:59:41.987 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 94489280518 2026-02-17T05:59:41.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:42.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280518 -lt 94489280518 2026-02-17T05:59:42.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:42.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051586 2026-02-17T05:59:42.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:42.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:59:42.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051586 2026-02-17T05:59:42.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:42.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T05:59:42.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-17T05:59:42.155 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 124554051586 2026-02-17T05:59:42.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:59:42.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T05:59:42.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:59:42.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:42.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:42.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:59:42.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:59:42.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:59:42.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:59:42.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:59:42.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:59:42.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:59:42.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:59:42.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:59:42.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:59:42.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:42.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:42.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:59:42.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:59:42.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:59:42.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T05:59:42.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T05:59:42.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:107: TEST_corrupt_and_repair_replicated: get_primary rbd SOMETHING 2026-02-17T05:59:42.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T05:59:42.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T05:59:42.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T05:59:42.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:107: TEST_corrupt_and_repair_replicated: corrupt_and_repair_one td/osd-scrub-repair rbd 1 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=rbd 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=1 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:59:42.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:59:42.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:59:42.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:59:42.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:59:42.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:59:42.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:59:42.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:59:44.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:59:44.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T05:59:44.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:59:44.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:59:44.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:59:44.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:59:44.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:59:44.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T05:59:44.943 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T05:59:45.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:59:45.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:59:45.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:59:45.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:59:45.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:59:45.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:59:45.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:59:45.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:59:45.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:59:45.483 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:59:45.483 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:59:45.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:59:45.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:59:45.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:59:45.485 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:59:45.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:59:45.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:59:45.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:59:45.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:59:45.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:59:45.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:59:45.504 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:45.489+0000 7fc4576eb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:45.506 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:45.493+0000 7fc4576eb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:45.507 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:45.493+0000 7fc4576eb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:59:45.634 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:59:45.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:45.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:45.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:46.731 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:46.717+0000 7fc4576eb8c0 -1 Falling back to public interface 2026-02-17T05:59:46.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:46.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:46.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T05:59:46.783 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T05:59:46.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:46.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:46.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:47.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:47.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:47.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T05:59:47.932 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T05:59:47.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:47.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:48.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:49.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:49.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:49.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T05:59:49.083 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T05:59:49.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:49.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:49.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T05:59:49.714 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:49.701+0000 7fc4576eb8c0 -1 osd.1 31 log_to_monitors true 2026-02-17T05:59:50.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T05:59:50.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:50.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T05:59:50.237 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T05:59:50.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:50.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 35 up_thru 35 down_at 32 last_clean_interval [29,31) [v2:127.0.0.1:6810/2248130946,v1:127.0.0.1:6811/2248130946] [v2:127.0.0.1:6812/2248130946,v1:127.0.0.1:6813/2248130946] exists,up 362b2a8d-5ec3-419d-8083-e6319212c2a2 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T05:59:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T05:59:50.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T05:59:50.386 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T05:59:50.387 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T05:59:50.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T05:59:50.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T05:59:50.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T05:59:50.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T05:59:50.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T05:59:50.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T05:59:50.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T05:59:50.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T05:59:50.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T05:59:50.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T05:59:50.636 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T05:59:50.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T05:59:50.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:50.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T05:59:50.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280521 2026-02-17T05:59:50.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280521 2026-02-17T05:59:50.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280521' 2026-02-17T05:59:50.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T05:59:50.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T05:59:50.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855362 2026-02-17T05:59:50.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855362 2026-02-17T05:59:50.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280521 1-150323855362' 2026-02-17T05:59:50.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:50.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-94489280521 2026-02-17T05:59:50.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:50.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T05:59:50.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-94489280521 2026-02-17T05:59:50.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:50.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280521 2026-02-17T05:59:50.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 94489280521' 2026-02-17T05:59:50.784 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 94489280521 2026-02-17T05:59:50.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:50.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280519 -lt 94489280521 2026-02-17T05:59:50.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T05:59:51.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T05:59:51.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T05:59:52.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280521 -lt 94489280521 2026-02-17T05:59:52.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T05:59:52.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-150323855362 2026-02-17T05:59:52.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T05:59:52.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T05:59:52.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-150323855362 2026-02-17T05:59:52.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T05:59:52.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855362 2026-02-17T05:59:52.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 150323855362' 2026-02-17T05:59:52.106 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 150323855362 2026-02-17T05:59:52.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T05:59:52.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855362 2026-02-17T05:59:52.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T05:59:52.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:52.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:52.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T05:59:52.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T05:59:52.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T05:59:52.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T05:59:52.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T05:59:52.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T05:59:52.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T05:59:52.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T05:59:52.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T05:59:52.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T05:59:52.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T05:59:52.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T05:59:52.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T05:59:52.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T05:59:52.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T05:59:52.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-17T05:59:52.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T05:59:52.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T05:59:52.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T05:59:52.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T05:59:52.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=1.3 2026-02-17T05:59:52.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 1.3 2026-02-17T05:59:52.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.3 2026-02-17T05:59:52.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.3 2026-02-17T05:59:52.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:52.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:52.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:52.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:53.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T05:59:30.018762+0000 2026-02-17T05:59:53.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.3 2026-02-17T05:59:53.220 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-17T05:59:53.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.3 2026-02-17T05:59:30.018762+0000 2026-02-17T05:59:53.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T05:59:53.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T05:59:30.018762+0000 2026-02-17T05:59:53.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T05:59:53.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T05:59:53.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:53.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:53.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:53.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:53.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:53.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:53.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:30.018762+0000 '>' 2026-02-17T05:59:30.018762+0000 2026-02-17T05:59:53.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:59:54.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:59:54.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:54.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:54.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:54.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:54.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:54.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:54.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:30.018762+0000 '>' 2026-02-17T05:59:30.018762+0000 2026-02-17T05:59:54.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:59:55.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:59:55.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:55.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:55.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:55.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:55.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:55.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:55.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:30.018762+0000 '>' 2026-02-17T05:59:30.018762+0000 2026-02-17T05:59:55.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T05:59:56.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T05:59:56.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T05:59:56.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T05:59:56.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T05:59:56.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T05:59:56.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T05:59:56.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T05:59:53.600237+0000 '>' 2026-02-17T05:59:30.018762+0000 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T05:59:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T05:59:56.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T05:59:56.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T05:59:56.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T05:59:56.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T05:59:56.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:59:58.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T05:59:58.457 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T05:59:58.458 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T05:59:58.458 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T05:59:58.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T05:59:58.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T05:59:58.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T05:59:58.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T05:59:58.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T05:59:58.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T05:59:58.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T05:59:58.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T05:59:58.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T05:59:58.997 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T05:59:58.998 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T05:59:58.998 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T05:59:59.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:59.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T05:59:59.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T05:59:59.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T05:59:59.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T05:59:59.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T05:59:59.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T05:59:59.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T05:59:59.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T05:59:59.016 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:59.001+0000 7fc32cdab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:59.017 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:59.005+0000 7fc32cdab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:59.018 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T05:59:59.005+0000 7fc32cdab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T05:59:59.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T05:59:59.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:00.255 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:00.241+0000 7fc32cdab8c0 -1 Falling back to public interface 2026-02-17T06:00:00.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:00.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:00.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:00:00.302 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:00:00.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:00.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:00.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:01.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:01.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:01.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:00:01.457 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:00:01.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:01.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:01.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:02.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:02.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:02.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:00:02.609 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:00:02.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:02.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:02.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:03.465 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:03.453+0000 7fc32cdab8c0 -1 osd.1 37 log_to_monitors true 2026-02-17T06:00:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:00:03.764 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:00:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [35,37) [v2:127.0.0.1:6810/2651046194,v1:127.0.0.1:6811/2651046194] [v2:127.0.0.1:6812/2651046194,v1:127.0.0.1:6813/2651046194] exists,up 362b2a8d-5ec3-419d-8083-e6319212c2a2 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:00:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:00:03.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:00:03.915 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:00:03.915 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:00:03.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:00:03.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:00:03.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:00:04.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:00:04.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:00:04.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:00:04.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:00:04.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:00:04.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:00:04.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:00:04.189 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:00:04.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:00:04.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:04.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:00:04.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280524 2026-02-17T06:00:04.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280524 2026-02-17T06:00:04.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280524' 2026-02-17T06:00:04.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:04.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:00:04.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659138 2026-02-17T06:00:04.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659138 2026-02-17T06:00:04.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280524 1-176093659138' 2026-02-17T06:00:04.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:04.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-94489280524 2026-02-17T06:00:04.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:04.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:00:04.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-94489280524 2026-02-17T06:00:04.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:04.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280524 2026-02-17T06:00:04.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 94489280524' 2026-02-17T06:00:04.334 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 94489280524 2026-02-17T06:00:04.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:04.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280523 -lt 94489280524 2026-02-17T06:00:04.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:00:05.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:00:05.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:05.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280523 -lt 94489280524 2026-02-17T06:00:05.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:00:06.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:00:06.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:06.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280525 -lt 94489280524 2026-02-17T06:00:06.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:06.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-176093659138 2026-02-17T06:00:06.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:06.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:00:06.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-176093659138 2026-02-17T06:00:06.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:06.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659138 2026-02-17T06:00:06.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 176093659138' 2026-02-17T06:00:06.804 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 176093659138 2026-02-17T06:00:06.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:00:06.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659138 -lt 176093659138 2026-02-17T06:00:06.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:00:06.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:06.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:07.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:00:07.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:00:07.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:00:07.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:00:07.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:00:07.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:00:07.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:00:07.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:00:07.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:00:07.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:00:07.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:07.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:07.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:00:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:00:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:00:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T06:00:07.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T06:00:07.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:00:07.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:00:07.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:00:07.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:00:07.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:00:07.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:00:07.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:00:07.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:00:07.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:00:07.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:00:07.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:00:07.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:00:07.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:00:07.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:00:07.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:00:07.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:00:07.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:00:07.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:00:07.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:00:07.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:00:07.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:00:07.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:00:07.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:00:07.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:00:07.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:07.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:07.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:00:07.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:00:07.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:00:07.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:00:07.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:00:07.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:00:07.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:00:07.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:00:07.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:00:07.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:00:07.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:00:07.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:00:07.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:00:07.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:00:07.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:00:07.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:00:07.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:00:07.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:00:07.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:00:07.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:00:07.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:00:07.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:00:07.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:07.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:07.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:00:07.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:00:07.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:00:07.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:00:07.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:00:07.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:07.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:07.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:00:07.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:00:07.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:00:07.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:00:07.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_erasure_appends td/osd-scrub-repair 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5746: TEST_corrupt_scrub_erasure_appends: corrupt_scrub_erasure td/osd-scrub-repair false 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3596: corrupt_scrub_erasure: local dir=td/osd-scrub-repair 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3597: corrupt_scrub_erasure: local allow_overwrites=false 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3598: corrupt_scrub_erasure: local poolname=ecpool 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3599: corrupt_scrub_erasure: local total_objs=7 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3601: corrupt_scrub_erasure: run_mon td/osd-scrub-repair a 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:00:07.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:00:07.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:00:07.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:00:07.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:00:07.875 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:00:07.875 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:07.875 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:07.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:07.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:00:07.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:00:07.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:00:07.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:00:07.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:00:07.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:00:07.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:00:07.906 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:00:07.906 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:00:07.906 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:00:07.907 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:00:07.907 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:07.907 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:07.907 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:00:07.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:00:07.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:00:07.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:00:07.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:00:07.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:00:07.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:00:07.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:00:07.962 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:00:07.962 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:00:07.962 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:00:07.963 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:00:07.963 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:07.963 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:07.964 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:00:07.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:00:07.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:00:08.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3602: corrupt_scrub_erasure: run_mgr td/osd-scrub-repair x 2026-02-17T06:00:08.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:00:08.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:00:08.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:00:08.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:00:08.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:00:08.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:00:08.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:00:08.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:00:08.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:00:08.114 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:00:08.114 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:08.114 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:08.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:08.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:00:08.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:00:08.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: seq 0 2 2026-02-17T06:00:08.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 0 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:00:08.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:00:08.138 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:00:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:00:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:00:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:00:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:00:08.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:00:08.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:00:08.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:00:08.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 3c55f2f2-59bb-463d-8f79-35cc0ab4c248' 2026-02-17T06:00:08.141 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:00:08.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:00:08.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDoA5RpsZVjCBAA64ESxp0v/V+B26vpQmfRHA== 2026-02-17T06:00:08.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDoA5RpsZVjCBAA64ESxp0v/V+B26vpQmfRHA=="}' 2026-02-17T06:00:08.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:00:08.237 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:00:08.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:00:08.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDoA5RpsZVjCBAA64ESxp0v/V+B26vpQmfRHA== --osd-uuid 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:00:08.260 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:08.245+0000 7f19ac71b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:08.262 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:08.249+0000 7f19ac71b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:08.263 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:08.249+0000 7f19ac71b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:08.263 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:08.249+0000 7f19ac71b8c0 -1 bdev(0x55ea8de85c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:00:08.263 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:08.249+0000 7f19ac71b8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:00:10.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:00:10.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:00:10.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:00:10.858 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:00:10.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:00:11.052 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:00:11.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:00:11.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:00:11.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:00:11.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:00:11.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:00:11.076 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:11.061+0000 7f91dcc0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:11.078 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:11.065+0000 7f91dcc0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:11.079 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:11.065+0000 7f91dcc0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:11.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:00:11.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:00:11.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:00:11.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:00:11.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:00:11.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:11.200 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:00:11.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:00:11.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:11.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:00:11.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:12.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:12.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:12.357 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:00:12.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:00:12.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:12.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:00:12.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:12.810 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:12.797+0000 7f91dcc0b8c0 -1 Falling back to public interface 2026-02-17T06:00:13.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:13.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:13.513 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:00:13.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:00:13.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:13.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:00:13.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:14.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:14.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:14.666 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:00:14.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:00:14.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:14.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:00:14.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:15.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:15.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:15.819 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:00:15.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:00:15.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:15.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:00:15.881 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:15.869+0000 7f91dcc0b8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:00:15.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:16.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:16.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:16.974 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:00:16.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:00:16.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:16.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:00:17.127 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/676051480,v1:127.0.0.1:6803/676051480] [v2:127.0.0.1:6804/676051480,v1:127.0.0.1:6805/676051480] exists,up 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:00:17.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:00:17.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:00:17.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 1 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:00:17.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:17.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:00:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:00:17.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:00:17.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:00:17.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 7d028e10-5f48-42cd-916e-29013ed1f617' 2026-02-17T06:00:17.135 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:00:17.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:00:17.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDxA5Rp9ggJCBAAFYhfaNTfA3TkQ2CCtSa+gg== 2026-02-17T06:00:17.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDxA5Rp9ggJCBAAFYhfaNTfA3TkQ2CCtSa+gg=="}' 2026-02-17T06:00:17.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7d028e10-5f48-42cd-916e-29013ed1f617 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:00:17.299 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:00:17.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:00:17.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDxA5Rp9ggJCBAAFYhfaNTfA3TkQ2CCtSa+gg== --osd-uuid 7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:00:17.326 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:17.313+0000 7f040e20b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:17.327 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:17.313+0000 7f040e20b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:17.328 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:17.317+0000 7f040e20b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:17.329 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:17.317+0000 7f040e20b8c0 -1 bdev(0x557f75571c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:00:17.329 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:17.317+0000 7f040e20b8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:00:19.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:00:19.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:00:19.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:00:19.583 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:00:19.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:00:19.776 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:00:19.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:00:19.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:00:19.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:00:19.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:00:19.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:00:19.800 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:19.785+0000 7efbfddcd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:19.801 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:19.789+0000 7efbfddcd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:19.803 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:19.789+0000 7efbfddcd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:19.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:00:19.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:00:19.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:00:19.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:00:19.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:00:19.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:19.926 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:00:19.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:00:19.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:19.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:20.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:21.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:21.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:21.081 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:00:21.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:00:21.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:21.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:21.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:21.319 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:21.305+0000 7efbfddcd8c0 -1 Falling back to public interface 2026-02-17T06:00:22.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:22.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:22.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:00:22.236 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:00:22.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:22.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:22.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:23.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:23.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:23.390 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:00:23.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:00:23.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:23.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:23.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:24.312 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:24.297+0000 7efbfddcd8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:00:24.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:24.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:24.543 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:00:24.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:00:24.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:24.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:24.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:25.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:25.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:25.694 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:00:25.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:00:25.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:25.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:25.847 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3711369425,v1:127.0.0.1:6811/3711369425] [v2:127.0.0.1:6812/3711369425,v1:127.0.0.1:6813/3711369425] exists,up 7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:00:25.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:00:25.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:00:25.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:00:25.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T06:00:25.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 2 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:00:25.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:00:25.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:00:25.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:00:25.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:00:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=480bd7f2-0e3f-4e12-baec-be7b73412268 2026-02-17T06:00:25.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 480bd7f2-0e3f-4e12-baec-be7b73412268' 2026-02-17T06:00:25.856 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 480bd7f2-0e3f-4e12-baec-be7b73412268 2026-02-17T06:00:25.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:00:25.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD5A5RpdVP7MhAAihXrSrIR3vomX1pkgH4yMg== 2026-02-17T06:00:25.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD5A5RpdVP7MhAAihXrSrIR3vomX1pkgH4yMg=="}' 2026-02-17T06:00:25.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 480bd7f2-0e3f-4e12-baec-be7b73412268 -i td/osd-scrub-repair/2/new.json 2026-02-17T06:00:26.022 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:00:26.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T06:00:26.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5A5RpdVP7MhAAihXrSrIR3vomX1pkgH4yMg== --osd-uuid 480bd7f2-0e3f-4e12-baec-be7b73412268 2026-02-17T06:00:26.049 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:26.037+0000 7f90224568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:26.050 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:26.037+0000 7f90224568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:26.051 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:26.037+0000 7f90224568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:26.052 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:26.037+0000 7f90224568c0 -1 bdev(0x5634839f5c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:00:26.052 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:26.037+0000 7f90224568c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T06:00:28.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T06:00:28.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:00:28.323 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:00:28.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:00:28.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:00:28.520 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:00:28.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:00:28.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:00:28.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:00:28.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:00:28.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:00:28.534 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:28.521+0000 7f84f232f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:28.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:28.521+0000 7f84f232f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:28.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:28.521+0000 7f84f232f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:28.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:00:28.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:00:28.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:00:28.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:00:28.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:00:28.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:28.669 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:00:28.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:00:28.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:28.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:00:28.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:29.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:29.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:29.824 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:00:29.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:00:29.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:29.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:00:29.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:30.311 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:30.297+0000 7f84f232f8c0 -1 Falling back to public interface 2026-02-17T06:00:30.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:30.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:30.978 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:00:30.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:00:30.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:30.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:00:31.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:32.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:32.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:32.130 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:00:32.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:00:32.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:32.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:00:32.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:33.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:33.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:33.288 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:00:33.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:00:33.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:33.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:00:33.344 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:33.329+0000 7f84f232f8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:00:33.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:34.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:34.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:34.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:00:34.445 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:00:34.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:34.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:00:34.596 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2319558054,v1:127.0.0.1:6819/2319558054] [v2:127.0.0.1:6820/2319558054,v1:127.0.0.1:6821/2319558054] exists,up 480bd7f2-0e3f-4e12-baec-be7b73412268 2026-02-17T06:00:34.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:00:34.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:00:34.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:00:34.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3606: corrupt_scrub_erasure: create_rbd_pool 2026-02-17T06:00:34.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:00:34.738 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:00:34.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:00:34.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:00:34.946 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:00:34.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:00:35.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:00:36.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3607: corrupt_scrub_erasure: create_pool foo 1 2026-02-17T06:00:36.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T06:00:36.436 INFO:tasks.workunit.client.0.trial074.stderr:pool 'foo' created 2026-02-17T06:00:36.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:00:37.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3609: corrupt_scrub_erasure: create_ec_pool ecpool false k=2 m=1 stripe_unit=2K --force 2026-02-17T06:00:37.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T06:00:37.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T06:00:37.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T06:00:37.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T06:00:37.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 stripe_unit=2K --force 2026-02-17T06:00:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T06:00:37.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T06:00:37.954 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T06:00:37.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:00:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T06:00:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T06:00:38.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:00:38.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:00:38.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:00:38.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:00:38.970 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:00:38.970 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:00:38.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:00:38.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:00:38.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:00:39.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:00:39.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:00:39.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:00:39.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:00:39.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:00:39.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:00:39.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:00:39.232 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:00:39.232 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:00:39.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:00:39.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:39.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:00:39.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T06:00:39.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T06:00:39.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T06:00:39.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:39.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:00:39.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T06:00:39.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T06:00:39.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964' 2026-02-17T06:00:39.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:39.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:00:39.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T06:00:39.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T06:00:39.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964 2-64424509442' 2026-02-17T06:00:39.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:39.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T06:00:39.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:39.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:00:39.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T06:00:39.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:39.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T06:00:39.451 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836486 2026-02-17T06:00:39.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T06:00:39.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:39.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T06:00:39.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:00:40.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:00:40.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:40.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T06:00:40.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:40.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T06:00:40.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:40.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:00:40.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T06:00:40.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:40.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T06:00:40.767 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T06:00:40.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T06:00:40.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:00:40.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T06:00:40.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:40.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T06:00:40.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:40.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:00:40.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T06:00:40.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:40.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T06:00:40.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T06:00:40.929 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509442 2026-02-17T06:00:40.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:00:41.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509443 -lt 64424509442 2026-02-17T06:00:41.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:00:41.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:41.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:41.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:00:41.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:00:41.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:00:41.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:00:41.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:00:41.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:00:41.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:00:41.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:00:41.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:00:41.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:00:41.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:41.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3610: corrupt_scrub_erasure: wait_for_clean 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:00:41.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:00:41.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:00:41.615 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:00:41.615 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:00:41.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:00:41.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:00:41.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:00:41.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:00:41.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:00:41.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:00:41.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:00:41.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:00:41.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:00:41.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:00:41.875 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:00:41.875 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:00:41.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:00:41.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:41.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:00:41.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T06:00:41.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T06:00:41.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T06:00:41.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:41.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:00:42.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T06:00:42.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T06:00:42.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672966' 2026-02-17T06:00:42.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:42.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:00:42.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T06:00:42.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T06:00:42.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672966 2-64424509444' 2026-02-17T06:00:42.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:42.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T06:00:42.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:42.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:00:42.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T06:00:42.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:42.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T06:00:42.093 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T06:00:42.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T06:00:42.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:42.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T06:00:42.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:00:43.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:00:43.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:43.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-17T06:00:43.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:43.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T06:00:43.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:43.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:00:43.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T06:00:43.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:43.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T06:00:43.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T06:00:43.415 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672966 2026-02-17T06:00:43.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:00:43.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T06:00:43.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:43.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T06:00:43.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:43.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:00:43.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T06:00:43.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:43.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T06:00:43.576 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509444 2026-02-17T06:00:43.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T06:00:43.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:00:43.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T06:00:43.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:00:43.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:43.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:43.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:00:43.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:00:43.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:00:43.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:00:43.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:00:43.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:00:43.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:00:43.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:00:44.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:00:44.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:00:44.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:44.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:44.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:00:44.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:00:44.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:00:44.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: seq 1 7 2026-02-17T06:00:44.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:00:44.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ1 2026-02-17T06:00:44.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ1 2026-02-17T06:00:44.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:00:44.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:00:44.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ1 2026-02-17T06:00:44.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:00:44.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:00:44.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:00:44.391 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:00:44.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:00:44.592 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:00:44.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:00:44.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:00:44.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:00:44.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 1 % 2 2026-02-17T06:00:44.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:00:44.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3621: corrupt_scrub_erasure: local payload=UVWXYZZZ 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3622: corrupt_scrub_erasure: echo UVWXYZZZ 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3623: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:00:44.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:00:44.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:00:44.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:00:44.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:00:44.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:00:44.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:00:44.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:00:45.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:47.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:00:47.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:00:47.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:00:47.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:00:47.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:00:47.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:00:47.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:00:47.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:00:47.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:00:47.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:00:47.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:00:47.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:00:47.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:00:47.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:00:47.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:00:47.130 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:00:47.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:00:47.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:00:47.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:00:47.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:00:47.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:00:47.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:00:47.149 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:47.140+0000 7fdb3e5228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:47.151 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:47.140+0000 7fdb3e5228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:47.152 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:47.140+0000 7fdb3e5228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:47.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:00:47.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:47.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:47.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:48.387 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:48.376+0000 7fdb3e5228c0 -1 Falling back to public interface 2026-02-17T06:00:48.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:48.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:48.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:00:48.432 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:00:48.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:48.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:48.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:49.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:49.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:49.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:00:49.581 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:00:49.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:49.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:49.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:50.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:50.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:50.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:00:50.731 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:00:50.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:50.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:50.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:00:51.391 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:51.380+0000 7fdb3e5228c0 -1 osd.1 33 log_to_monitors true 2026-02-17T06:00:51.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:00:51.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:51.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:00:51.878 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:00:51.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:51.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:00:52.025 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [10,33) [v2:127.0.0.1:6810/416079090,v1:127.0.0.1:6811/416079090] [v2:127.0.0.1:6812/416079090,v1:127.0.0.1:6813/416079090] exists,up 7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:00:52.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:00:52.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:00:52.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:00:52.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:00:52.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:00:52.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:00:52.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:00:52.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:00:52.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:00:52.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:00:52.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:00:52.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:00:52.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:00:52.296 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:00:52.296 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:00:52.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:00:52.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:52.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:00:52.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T06:00:52.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T06:00:52.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T06:00:52.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:52.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:00:52.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789954 2026-02-17T06:00:52.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789954 2026-02-17T06:00:52.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-158913789954' 2026-02-17T06:00:52.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:00:52.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:00:52.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509447 2026-02-17T06:00:52.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509447 2026-02-17T06:00:52.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-158913789954 2-64424509447' 2026-02-17T06:00:52.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:52.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T06:00:52.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:52.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:00:52.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T06:00:52.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:52.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T06:00:52.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T06:00:52.511 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836491 2026-02-17T06:00:52.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:52.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T06:00:52.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:00:53.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:00:53.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:53.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T06:00:53.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:00:54.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:00:54.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:00:54.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T06:00:54.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:54.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789954 2026-02-17T06:00:54.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:54.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:00:54.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789954 2026-02-17T06:00:54.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:54.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789954 2026-02-17T06:00:54.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789954' 2026-02-17T06:00:54.976 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 158913789954 2026-02-17T06:00:54.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:00:55.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789954 -lt 158913789954 2026-02-17T06:00:55.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:00:55.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509447 2026-02-17T06:00:55.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:00:55.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:00:55.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509447 2026-02-17T06:00:55.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:00:55.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509447 2026-02-17T06:00:55.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-17T06:00:55.139 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509447 2026-02-17T06:00:55.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:00:55.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509447 -lt 64424509447 2026-02-17T06:00:55.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:00:55.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:55.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:55.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:00:55.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:00:55.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:00:55.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:00:55.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:00:55.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:00:55.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:00:55.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:00:55.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:00:55.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:00:55.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:00:55.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:00:55.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:00:55.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:00:55.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:00:55.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:00:55.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ2 2026-02-17T06:00:55.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ2 2026-02-17T06:00:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:00:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:00:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ2 2026-02-17T06:00:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:00:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:00:55.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:00:56.005 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:00:56.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:00:56.207 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:00:56.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:00:56.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:00:56.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:00:56.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 2 % 2 2026-02-17T06:00:56.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T06:00:56.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:00:56.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3628: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=1 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:2048 bytes (2.0 kB, 2.0 KiB) copied, 9.5087e-05 s, 21.5 MB/s 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3629: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:56.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:00:56.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:00:57.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:00:58.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:00:58.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:00:58.746 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:00:58.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:58.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:00:58.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:00:58.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:00:58.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:00:58.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:00:58.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:00:58.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:00:58.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:00:58.770 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:58.760+0000 7fdbb5c248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:58.771 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:58.760+0000 7fdbb5c248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:58.772 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:00:58.760+0000 7fdbb5c248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:00:58.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:00:58.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:00:59.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:00.043 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:00.032+0000 7fdbb5c248c0 -1 Falling back to public interface 2026-02-17T06:01:00.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:00.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:00.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:01:00.052 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:00.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:00.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:00.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:01.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:01.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:01.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:01:01.205 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:01:01.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:01.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:01.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:02.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:02.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:02.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:01:02.358 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:01:02.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:02.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:02.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:03.038 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:03.028+0000 7fdbb5c248c0 -1 osd.0 40 log_to_monitors true 2026-02-17T06:01:03.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:03.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:03.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:01:03.513 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:01:03.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:03.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:03.664 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 44 up_thru 44 down_at 41 last_clean_interval [5,40) [v2:127.0.0.1:6802/263120714,v1:127.0.0.1:6803/263120714] [v2:127.0.0.1:6804/263120714,v1:127.0.0.1:6805/263120714] exists,up 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:01:03.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:01:03.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:01:03.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:01:03.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:01:03.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:01:03.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:01:03.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:01:03.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:01:03.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:01:03.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:01:03.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:01:03.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:01:03.934 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:03.934 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:01:03.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:01:03.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:03.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:01:04.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T06:01:04.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T06:01:04.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561026' 2026-02-17T06:01:04.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:04.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:01:04.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789957 2026-02-17T06:01:04.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789957 2026-02-17T06:01:04.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561026 1-158913789957' 2026-02-17T06:01:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:04.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:01:04.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509450 2026-02-17T06:01:04.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509450 2026-02-17T06:01:04.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561026 1-158913789957 2-64424509450' 2026-02-17T06:01:04.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:04.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-188978561026 2026-02-17T06:01:04.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:04.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:01:04.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-188978561026 2026-02-17T06:01:04.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:04.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T06:01:04.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 188978561026' 2026-02-17T06:01:04.156 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 188978561026 2026-02-17T06:01:04.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:04.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 188978561026 2026-02-17T06:01:04.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:01:05.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:01:05.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:05.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T06:01:05.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:05.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789957 2026-02-17T06:01:05.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:05.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:01:05.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789957 2026-02-17T06:01:05.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:05.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789957 2026-02-17T06:01:05.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789957' 2026-02-17T06:01:05.473 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 158913789957 2026-02-17T06:01:05.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:01:05.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789957 -lt 158913789957 2026-02-17T06:01:05.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:05.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509450 2026-02-17T06:01:05.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:05.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:01:05.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509450 2026-02-17T06:01:05.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:05.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509450 2026-02-17T06:01:05.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2026-02-17T06:01:05.640 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509450 2026-02-17T06:01:05.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:01:05.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509450 -lt 64424509450 2026-02-17T06:01:05.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:01:05.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:05.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:05.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:01:05.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:01:05.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:01:05.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:01:05.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:01:05.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:01:05.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:01:05.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:01:06.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:01:06.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:01:06.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:06.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ3 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ3 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:01:06.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:01:06.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ3 2026-02-17T06:01:06.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:01:06.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:01:06.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:01:06.501 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:01:06.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:01:06.704 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:01:06.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:01:06.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:01:06.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:01:06.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 3 % 2 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3634: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T06:01:06.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:01:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:01:08.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:01:08.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T06:01:08.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:01:08.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:01:08.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:01:08.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:01:08.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:01:08.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ3 remove 2026-02-17T06:01:08.673 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#3:b197b25d:::EOBJ3:head# 2026-02-17T06:01:09.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:01:09.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:01:09.210 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:01:09.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:01:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:01:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:01:09.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:01:09.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:01:09.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:01:09.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:01:09.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:01:09.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:01:09.234 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:09.224+0000 7fa3766358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:09.235 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:09.224+0000 7fa3766358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:09.236 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:09.224+0000 7fa3766358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:09.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:01:09.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:09.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:09.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:10.482 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:10.472+0000 7fa3766358c0 -1 Falling back to public interface 2026-02-17T06:01:10.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:10.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:10.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:01:10.519 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:10.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:10.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:11.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:11.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:11.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:01:11.669 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:01:11.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:11.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:11.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:12.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:12.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:12.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:01:12.821 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:01:12.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:12.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:12.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:13.451 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:13.440+0000 7fa3766358c0 -1 osd.1 47 log_to_monitors true 2026-02-17T06:01:13.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:13.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:13.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:01:13.974 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:01:13.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:13.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:14.126 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 48 last_clean_interval [37,47) [v2:127.0.0.1:6810/1517599240,v1:127.0.0.1:6811/1517599240] [v2:127.0.0.1:6812/1517599240,v1:127.0.0.1:6813/1517599240] exists,up 7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:01:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:01:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:01:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:01:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:01:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:01:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:01:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:01:14.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:01:14.127 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:01:14.127 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:01:14.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:01:14.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:01:14.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:01:14.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:01:14.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:01:14.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:01:14.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:01:14.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:01:14.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:01:14.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:01:14.384 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:14.385 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:01:14.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:01:14.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:01:14.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561029 2026-02-17T06:01:14.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561029 2026-02-17T06:01:14.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561029' 2026-02-17T06:01:14.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:14.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:01:14.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332098 2026-02-17T06:01:14.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332098 2026-02-17T06:01:14.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561029 1-219043332098' 2026-02-17T06:01:14.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:14.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:01:14.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509454 2026-02-17T06:01:14.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509454 2026-02-17T06:01:14.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561029 1-219043332098 2-64424509454' 2026-02-17T06:01:14.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:14.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-188978561029 2026-02-17T06:01:14.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:14.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:01:14.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-188978561029 2026-02-17T06:01:14.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:14.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561029 2026-02-17T06:01:14.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 188978561029' 2026-02-17T06:01:14.599 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 188978561029 2026-02-17T06:01:14.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:14.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561028 -lt 188978561029 2026-02-17T06:01:14.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:01:15.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:01:15.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:15.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561028 -lt 188978561029 2026-02-17T06:01:15.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:01:16.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:01:16.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:17.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 188978561029 2026-02-17T06:01:17.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:17.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332098 2026-02-17T06:01:17.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:17.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:01:17.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332098 2026-02-17T06:01:17.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:17.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332098 2026-02-17T06:01:17.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332098' 2026-02-17T06:01:17.064 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 219043332098 2026-02-17T06:01:17.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:01:17.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332098 -lt 219043332098 2026-02-17T06:01:17.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:17.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509454 2026-02-17T06:01:17.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:17.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:01:17.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509454 2026-02-17T06:01:17.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:17.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509454 2026-02-17T06:01:17.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2026-02-17T06:01:17.226 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509454 2026-02-17T06:01:17.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:01:17.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509454 -lt 64424509454 2026-02-17T06:01:17.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:01:17.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:17.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:17.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:01:17.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:01:17.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:01:17.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:01:17.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:01:17.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:01:17.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:01:17.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:01:17.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:01:17.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:01:17.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:17.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:17.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:01:17.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:01:17.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:01:17.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ4 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ4 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ4 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:01:17.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:01:18.091 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:01:18.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:01:18.296 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:01:18.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:01:18.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:01:18.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:01:18.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 4 % 2 2026-02-17T06:01:18.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T06:01:18.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:01:18.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3638: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key1-EOBJ4 val1-EOBJ4 2026-02-17T06:01:18.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3639: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key2-EOBJ4 val2-EOBJ4 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3642: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3643: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:01:18.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:01:18.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:01:19.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:01:19.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:01:19.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:01:19.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:01:19.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:01:19.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:01:19.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:01:19.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:01:20.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:01:20.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:01:20.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:01:20.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:01:20.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:01:20.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:01:20.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:01:20.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:01:20.875 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:01:20.875 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:01:20.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:01:20.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:01:20.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:01:20.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:01:20.877 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:01:20.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:01:20.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:01:20.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:01:20.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:01:20.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:01:20.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:01:20.893 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:20.880+0000 7ff1a0beb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:20.894 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:20.884+0000 7ff1a0beb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:20.895 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:20.884+0000 7ff1a0beb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:01:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:21.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:22.131 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:22.120+0000 7ff1a0beb8c0 -1 Falling back to public interface 2026-02-17T06:01:22.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:22.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:22.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:01:22.185 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:22.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:22.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:22.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:23.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:23.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:23.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:01:23.338 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:01:23.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:23.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:23.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:24.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:24.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:24.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:01:24.493 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:01:24.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:24.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:24.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:25.365 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:25.356+0000 7ff1a0beb8c0 -1 osd.0 54 log_to_monitors true 2026-02-17T06:01:25.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:25.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:25.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:01:25.649 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:01:25.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:25.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:01:25.802 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 58 up_thru 58 down_at 55 last_clean_interval [44,54) [v2:127.0.0.1:6802/1611175190,v1:127.0.0.1:6803/1611175190] [v2:127.0.0.1:6804/1611175190,v1:127.0.0.1:6805/1611175190] exists,up 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:01:25.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:01:25.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:01:25.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:01:25.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:01:25.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:01:25.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:01:25.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:01:25.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:01:25.804 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:01:25.804 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:01:25.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:01:25.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:01:25.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:01:25.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:01:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:01:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:01:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:01:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:01:25.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:01:26.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:01:26.059 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:26.060 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:01:26.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:01:26.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:26.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:01:26.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103170 2026-02-17T06:01:26.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103170 2026-02-17T06:01:26.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103170' 2026-02-17T06:01:26.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:26.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:01:26.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332101 2026-02-17T06:01:26.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332101 2026-02-17T06:01:26.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103170 1-219043332101' 2026-02-17T06:01:26.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:26.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:01:26.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509457 2026-02-17T06:01:26.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509457 2026-02-17T06:01:26.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103170 1-219043332101 2-64424509457' 2026-02-17T06:01:26.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:26.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103170 2026-02-17T06:01:26.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:26.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:01:26.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103170 2026-02-17T06:01:26.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:26.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103170 2026-02-17T06:01:26.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103170' 2026-02-17T06:01:26.277 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 249108103170 2026-02-17T06:01:26.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:26.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103170 -lt 249108103170 2026-02-17T06:01:26.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:26.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332101 2026-02-17T06:01:26.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:26.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:01:26.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332101 2026-02-17T06:01:26.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332101 2026-02-17T06:01:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332101' 2026-02-17T06:01:26.440 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 219043332101 2026-02-17T06:01:26.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:01:26.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332101 -lt 219043332101 2026-02-17T06:01:26.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:26.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509457 2026-02-17T06:01:26.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:26.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:01:26.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509457 2026-02-17T06:01:26.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:26.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509457 2026-02-17T06:01:26.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509457' 2026-02-17T06:01:26.604 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509457 2026-02-17T06:01:26.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:01:26.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509457 -lt 64424509457 2026-02-17T06:01:26.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:01:26.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:26.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:26.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:01:26.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:01:26.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:01:26.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:01:26.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:01:26.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:01:26.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:01:26.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:01:27.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:01:27.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:01:27.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:27.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:27.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3644: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:01:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:01:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:01:28.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:01:28.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:01:28.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:01:28.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:01:28.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:01:28.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:01:28.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:01:28.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:01:29.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T06:01:29.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:01:29.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:01:29.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:01:29.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:01:29.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:01:29.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:01:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:01:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:01:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:01:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:01:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:01:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:01:29.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:01:29.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:01:29.766 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:01:29.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:01:29.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:01:29.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:01:29.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:01:29.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:01:29.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:01:29.786 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:29.776+0000 7f970cbde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:29.787 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:29.776+0000 7f970cbde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:29.789 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:29.776+0000 7f970cbde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:01:29.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:29.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:30.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:31.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:31.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:31.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:01:31.072 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:31.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:31.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:31.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:31.263 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:31.252+0000 7f970cbde8c0 -1 Falling back to public interface 2026-02-17T06:01:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:01:32.225 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:01:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:32.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:33.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:33.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:33.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:01:33.375 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:01:33.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:33.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:33.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:34.234 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:34.224+0000 7f970cbde8c0 -1 osd.2 59 log_to_monitors true 2026-02-17T06:01:34.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:34.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:34.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:01:34.532 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:01:34.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:34.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:34.679 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [15,59) [v2:127.0.0.1:6818/3174810230,v1:127.0.0.1:6819/3174810230] [v2:127.0.0.1:6820/3174810230,v1:127.0.0.1:6821/3174810230] exists,up 480bd7f2-0e3f-4e12-baec-be7b73412268 2026-02-17T06:01:34.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:01:34.680 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:01:34.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:01:34.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:01:34.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:01:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:01:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:01:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:01:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:01:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:01:34.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:01:34.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:01:34.954 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:34.954 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:01:34.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:01:34.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:34.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:01:35.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103172 2026-02-17T06:01:35.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103172 2026-02-17T06:01:35.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103172' 2026-02-17T06:01:35.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:35.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:01:35.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332104 2026-02-17T06:01:35.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332104 2026-02-17T06:01:35.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103172 1-219043332104' 2026-02-17T06:01:35.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:35.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:01:35.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939650 2026-02-17T06:01:35.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939650 2026-02-17T06:01:35.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103172 1-219043332104 2-270582939650' 2026-02-17T06:01:35.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:35.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103172 2026-02-17T06:01:35.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:35.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:01:35.174 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103172 2026-02-17T06:01:35.174 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:35.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103172 2026-02-17T06:01:35.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103172' 2026-02-17T06:01:35.176 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 249108103172 2026-02-17T06:01:35.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:35.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103171 -lt 249108103172 2026-02-17T06:01:35.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:01:36.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:01:36.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:36.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103173 -lt 249108103172 2026-02-17T06:01:36.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:36.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332104 2026-02-17T06:01:36.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:36.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:01:36.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332104 2026-02-17T06:01:36.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:36.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332104 2026-02-17T06:01:36.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332104' 2026-02-17T06:01:36.496 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 219043332104 2026-02-17T06:01:36.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:01:36.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332104 -lt 219043332104 2026-02-17T06:01:36.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:36.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-270582939650 2026-02-17T06:01:36.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:36.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:01:36.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-270582939650 2026-02-17T06:01:36.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:36.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939650 2026-02-17T06:01:36.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 270582939650' 2026-02-17T06:01:36.650 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 270582939650 2026-02-17T06:01:36.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:01:36.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 270582939650 -lt 270582939650 2026-02-17T06:01:36.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:01:36.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:36.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:36.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:01:36.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:01:36.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:01:36.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:01:36.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:01:36.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:01:36.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:01:36.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:01:37.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:01:37.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:01:37.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:37.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:37.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3645: corrupt_scrub_erasure: echo -n val3-EOBJ4 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3646: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:01:37.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:01:37.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:01:38.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:01:38.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:01:38.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:01:38.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:01:38.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:01:38.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:01:38.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:01:38.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:01:39.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:01:39.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:01:39.810 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:01:39.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:39.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:39.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:01:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:01:39.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:01:39.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:01:39.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:01:39.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:01:39.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:01:39.834 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:39.823+0000 7f2077faa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:39.835 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:39.823+0000 7f2077faa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:39.836 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:39.823+0000 7f2077faa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:01:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:01:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:01:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:01:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:01:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:01:39.964 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:01:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:40.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:41.099 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:41.087+0000 7f2077faa8c0 -1 Falling back to public interface 2026-02-17T06:01:41.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:41.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:41.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:01:41.121 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:41.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:41.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:41.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:42.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:42.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:42.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:01:42.276 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:01:42.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:42.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:42.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:01:43.431 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:01:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:43.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:43.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:44.114 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:44.103+0000 7f2077faa8c0 -1 osd.2 64 log_to_monitors true 2026-02-17T06:01:44.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:44.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:44.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:01:44.586 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:01:44.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:44.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:01:44.734 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [63,64) [v2:127.0.0.1:6818/3996728514,v1:127.0.0.1:6819/3996728514] [v2:127.0.0.1:6820/3996728514,v1:127.0.0.1:6821/3996728514] exists,up 480bd7f2-0e3f-4e12-baec-be7b73412268 2026-02-17T06:01:44.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:01:44.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:01:44.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:01:44.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:01:44.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:01:44.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:01:44.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:01:44.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:01:44.736 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:01:44.736 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:01:44.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:01:44.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:01:44.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:01:44.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:01:44.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:01:44.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:01:44.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:01:44.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:01:44.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:01:44.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:01:44.993 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:44.993 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:01:44.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:01:44.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:44.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:01:45.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103175 2026-02-17T06:01:45.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103175 2026-02-17T06:01:45.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103175' 2026-02-17T06:01:45.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:45.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:01:45.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332107 2026-02-17T06:01:45.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332107 2026-02-17T06:01:45.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103175 1-219043332107' 2026-02-17T06:01:45.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:45.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:01:45.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776130 2026-02-17T06:01:45.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776130 2026-02-17T06:01:45.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103175 1-219043332107 2-292057776130' 2026-02-17T06:01:45.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:45.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103175 2026-02-17T06:01:45.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:45.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:01:45.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103175 2026-02-17T06:01:45.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:45.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103175 2026-02-17T06:01:45.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103175' 2026-02-17T06:01:45.211 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 249108103175 2026-02-17T06:01:45.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:45.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103174 -lt 249108103175 2026-02-17T06:01:45.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:01:46.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:01:46.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:46.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103176 -lt 249108103175 2026-02-17T06:01:46.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:46.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332107 2026-02-17T06:01:46.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:46.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:01:46.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332107 2026-02-17T06:01:46.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:46.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332107 2026-02-17T06:01:46.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332107' 2026-02-17T06:01:46.521 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 219043332107 2026-02-17T06:01:46.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:01:46.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332107 -lt 219043332107 2026-02-17T06:01:46.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:46.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776130 2026-02-17T06:01:46.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:46.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:01:46.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776130 2026-02-17T06:01:46.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:46.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776130 2026-02-17T06:01:46.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776130' 2026-02-17T06:01:46.684 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 292057776130 2026-02-17T06:01:46.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:01:46.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776130 -lt 292057776130 2026-02-17T06:01:46.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:01:46.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:46.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:47.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:01:47.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:01:47.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:01:47.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:01:47.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:01:47.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:01:47.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:01:47.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:01:47.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:01:47.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:01:47.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:47.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:47.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:01:47.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:01:47.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:01:47.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3647: corrupt_scrub_erasure: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-17T06:01:47.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:01:47.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ5 2026-02-17T06:01:47.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ5 2026-02-17T06:01:47.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:01:47.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:01:47.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ5 2026-02-17T06:01:47.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:01:47.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:01:47.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:01:47.575 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:01:47.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:01:47.778 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:01:47.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:01:47.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:01:47.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:01:47.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 5 % 2 2026-02-17T06:01:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:01:47.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:01:47.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3652: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=2 2026-02-17T06:01:47.822 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records in 2026-02-17T06:01:47.822 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records out 2026-02-17T06:01:47.822 INFO:tasks.workunit.client.0.trial074.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000100337 s, 40.8 MB/s 2026-02-17T06:01:47.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3653: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:01:47.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:01:47.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:01:47.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:01:47.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:01:49.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:01:49.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:01:49.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:01:49.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:01:49.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:01:49.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:01:49.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:01:49.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:01:50.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:01:50.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:01:50.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:01:50.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:01:50.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:01:50.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:01:50.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:01:50.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:01:50.329 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:01:50.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:01:50.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:01:50.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:01:50.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:01:50.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:01:50.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:01:50.339 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:50.327+0000 7f78405ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:50.340 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:50.327+0000 7f78405ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:50.341 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:50.331+0000 7f78405ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:01:50.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:01:50.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:50.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:50.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:01:51.626 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:51.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:51.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:51.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:51.819 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:51.807+0000 7f78405ba8c0 -1 Falling back to public interface 2026-02-17T06:01:52.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:52.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:52.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:01:52.775 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:01:52.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:52.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:52.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:53.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:53.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:53.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:01:53.925 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:01:53.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:53.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:54.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:01:54.803 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:01:54.791+0000 7f78405ba8c0 -1 osd.1 71 log_to_monitors true 2026-02-17T06:01:55.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:01:55.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:01:55.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:01:55.079 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:01:55.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:01:55.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 75 up_thru 75 down_at 72 last_clean_interval [51,71) [v2:127.0.0.1:6810/1325884629,v1:127.0.0.1:6811/1325884629] [v2:127.0.0.1:6812/1325884629,v1:127.0.0.1:6813/1325884629] exists,up 7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:01:55.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:01:55.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:01:55.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:01:55.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:01:55.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:01:55.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:01:55.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:01:55.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:01:55.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:01:55.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:01:55.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:01:55.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:01:55.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:01:55.492 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:01:55.492 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:01:55.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:01:55.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:55.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:01:55.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103179 2026-02-17T06:01:55.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103179 2026-02-17T06:01:55.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103179' 2026-02-17T06:01:55.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:55.563 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:01:55.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=322122547202 2026-02-17T06:01:55.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 322122547202 2026-02-17T06:01:55.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103179 1-322122547202' 2026-02-17T06:01:55.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:01:55.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:01:55.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776133 2026-02-17T06:01:55.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776133 2026-02-17T06:01:55.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103179 1-322122547202 2-292057776133' 2026-02-17T06:01:55.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:55.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103179 2026-02-17T06:01:55.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:55.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:01:55.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103179 2026-02-17T06:01:55.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:55.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103179 2026-02-17T06:01:55.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103179' 2026-02-17T06:01:55.711 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 249108103179 2026-02-17T06:01:55.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:55.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103177 -lt 249108103179 2026-02-17T06:01:55.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:01:56.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:01:56.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:01:57.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103179 -lt 249108103179 2026-02-17T06:01:57.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:57.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-322122547202 2026-02-17T06:01:57.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:57.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:01:57.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-322122547202 2026-02-17T06:01:57.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:57.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=322122547202 2026-02-17T06:01:57.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 322122547202' 2026-02-17T06:01:57.034 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 322122547202 2026-02-17T06:01:57.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:01:57.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 322122547202 -lt 322122547202 2026-02-17T06:01:57.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:01:57.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776133 2026-02-17T06:01:57.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:01:57.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:01:57.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776133 2026-02-17T06:01:57.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:01:57.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776133 2026-02-17T06:01:57.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776133' 2026-02-17T06:01:57.189 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 292057776133 2026-02-17T06:01:57.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:01:57.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776133 -lt 292057776133 2026-02-17T06:01:57.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:01:57.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:57.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:57.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:01:57.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:01:57.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:01:57.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:01:57.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:01:57.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:01:57.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:01:57.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:01:57.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:01:57.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:01:57.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:01:57.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:01:57.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:01:57.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:01:57.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:01:57.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:01:57.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ6 2026-02-17T06:01:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ6 2026-02-17T06:01:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:01:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:01:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ6 2026-02-17T06:01:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:01:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:01:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:01:58.065 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:01:58.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:01:58.270 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:01:58.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:01:58.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:01:58.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:01:58.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 6 % 2 2026-02-17T06:01:58.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T06:01:58.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:01:58.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3657: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:01:58.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:01:58.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:01:58.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:01:58.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:01:58.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:01:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:01:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:01:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:01:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:01:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:01:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:01:59.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:01:59.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:02:00.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:02:00.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:02:00.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:02:00.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:00.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:02:00.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:02:00.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:02:00.814 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:00.803+0000 7f8c84f468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:00.815 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:00.803+0000 7f8c84f468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:00.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:00.803+0000 7f8c84f468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:02:00.945 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:02:00.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:00.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:01.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:02.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:02.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:02.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:02:02.103 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:02.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:02.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:02.115 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:02.103+0000 7f8c84f468c0 -1 Falling back to public interface 2026-02-17T06:02:02.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:03.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:03.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:03.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:02:03.254 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:02:03.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:03.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:03.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:04.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:04.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:04.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:02:04.405 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:02:04.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:04.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:04.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:05.098 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:05.087+0000 7f8c84f468c0 -1 osd.0 78 log_to_monitors true 2026-02-17T06:02:05.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:05.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:05.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:02:05.555 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:02:05.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:05.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [58,78) [v2:127.0.0.1:6802/1573339992,v1:127.0.0.1:6803/1573339992] [v2:127.0.0.1:6804/1573339992,v1:127.0.0.1:6805/1573339992] exists,up 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:02:05.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:02:05.707 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:02:05.707 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:02:05.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:02:05.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:02:05.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:02:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:02:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:02:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:02:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:02:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:02:05.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:02:05.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:02:05.965 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:05.965 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:02:05.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:02:05.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:05.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:02:06.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318274 2026-02-17T06:02:06.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318274 2026-02-17T06:02:06.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274' 2026-02-17T06:02:06.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:06.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:02:06.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=322122547205 2026-02-17T06:02:06.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 322122547205 2026-02-17T06:02:06.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274 1-322122547205' 2026-02-17T06:02:06.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:06.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:02:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776136 2026-02-17T06:02:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776136 2026-02-17T06:02:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274 1-322122547205 2-292057776136' 2026-02-17T06:02:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:06.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318274 2026-02-17T06:02:06.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:06.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:02:06.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318274 2026-02-17T06:02:06.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:06.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318274 2026-02-17T06:02:06.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318274' 2026-02-17T06:02:06.181 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 352187318274 2026-02-17T06:02:06.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:06.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103179 -lt 352187318274 2026-02-17T06:02:06.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:07.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:02:07.340 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:07.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318274 -lt 352187318274 2026-02-17T06:02:07.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:07.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-322122547205 2026-02-17T06:02:07.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:07.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:02:07.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-322122547205 2026-02-17T06:02:07.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:07.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=322122547205 2026-02-17T06:02:07.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 322122547205' 2026-02-17T06:02:07.496 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 322122547205 2026-02-17T06:02:07.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:02:07.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 322122547205 -lt 322122547205 2026-02-17T06:02:07.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:07.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776136 2026-02-17T06:02:07.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:07.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:02:07.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776136 2026-02-17T06:02:07.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:07.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776136 2026-02-17T06:02:07.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776136' 2026-02-17T06:02:07.659 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 292057776136 2026-02-17T06:02:07.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:07.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776136 -lt 292057776136 2026-02-17T06:02:07.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:02:07.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:07.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:08.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:02:08.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:02:08.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:02:08.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:02:08.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:02:08.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:02:08.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:02:08.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:02:08.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:02:08.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:02:08.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:08.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:08.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:02:08.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:02:08.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:02:08.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3658: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T06:02:08.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3659: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:02:08.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:02:08.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:02:08.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:02:08.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:02:08.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:02:08.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:02:08.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:02:08.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:02:09.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:02:09.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:02:09.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:02:09.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:02:09.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:02:09.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:02:09.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:02:09.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:02:10.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:02:10.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:02:10.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:02:10.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:02:10.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:02:10.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:02:10.828 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:02:10.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:02:10.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:02:10.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:02:10.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:02:10.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:02:10.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:02:10.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:02:10.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:02:10.831 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:02:10.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:02:10.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:02:10.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:02:10.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:02:10.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:02:10.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:02:10.850 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:10.839+0000 7fbf117438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:10.852 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:10.839+0000 7fbf117438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:10.853 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:10.839+0000 7fbf117438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:10.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:10.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:02:11.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:12.111 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:12.099+0000 7fbf117438c0 -1 Falling back to public interface 2026-02-17T06:02:12.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:12.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:12.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:02:12.136 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:12.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:12.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:02:12.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:13.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:13.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:13.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:02:13.293 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:02:13.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:13.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:02:13.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:14.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:14.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:14.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:02:14.447 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:02:14.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:14.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:02:14.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:15.111 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:15.099+0000 7fbf117438c0 -1 osd.1 83 log_to_monitors true 2026-02-17T06:02:15.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:15.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:15.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:02:15.602 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:02:15.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:15.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:02:15.752 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 87 up_thru 87 down_at 84 last_clean_interval [75,83) [v2:127.0.0.1:6810/2274050429,v1:127.0.0.1:6811/2274050429] [v2:127.0.0.1:6812/2274050429,v1:127.0.0.1:6813/2274050429] exists,up 7d028e10-5f48-42cd-916e-29013ed1f617 2026-02-17T06:02:15.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:02:15.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:02:15.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:02:15.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:02:15.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:02:15.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:02:15.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:02:15.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:02:15.753 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:02:15.753 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:02:15.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:02:15.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:02:15.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:02:15.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:02:15.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:02:15.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:02:15.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:02:15.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:02:15.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:02:16.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:02:16.021 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:16.022 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:02:16.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:02:16.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:16.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:02:16.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318277 2026-02-17T06:02:16.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318277 2026-02-17T06:02:16.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277' 2026-02-17T06:02:16.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:16.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:02:16.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154754 2026-02-17T06:02:16.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154754 2026-02-17T06:02:16.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277 1-373662154754' 2026-02-17T06:02:16.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:16.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:02:16.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776139 2026-02-17T06:02:16.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776139 2026-02-17T06:02:16.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277 1-373662154754 2-292057776139' 2026-02-17T06:02:16.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:16.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318277 2026-02-17T06:02:16.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:16.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:02:16.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318277 2026-02-17T06:02:16.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:16.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318277 2026-02-17T06:02:16.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318277' 2026-02-17T06:02:16.239 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 352187318277 2026-02-17T06:02:16.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:16.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318275 -lt 352187318277 2026-02-17T06:02:16.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:17.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:02:17.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:17.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318277 -lt 352187318277 2026-02-17T06:02:17.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:17.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154754 2026-02-17T06:02:17.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:17.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:02:17.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154754 2026-02-17T06:02:17.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:17.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154754 2026-02-17T06:02:17.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154754' 2026-02-17T06:02:17.552 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 373662154754 2026-02-17T06:02:17.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:02:17.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154754 -lt 373662154754 2026-02-17T06:02:17.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:17.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776139 2026-02-17T06:02:17.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:17.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:02:17.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776139 2026-02-17T06:02:17.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:17.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776139 2026-02-17T06:02:17.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776139' 2026-02-17T06:02:17.717 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 292057776139 2026-02-17T06:02:17.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:17.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776139 -lt 292057776139 2026-02-17T06:02:17.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:02:17.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:17.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:18.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:02:18.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:02:18.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:02:18.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:02:18.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:02:18.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:02:18.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:02:18.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:02:18.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:02:18.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:02:18.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:18.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ7 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ7 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ7 2026-02-17T06:02:18.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:02:18.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:02:18.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:02:18.595 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:02:18.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:02:18.800 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:02:18.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:02:18.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:02:18.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:02:18.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 7 % 2 2026-02-17T06:02:18.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:02:18.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:02:18.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3663: corrupt_scrub_erasure: local payload=MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T06:02:18.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3664: corrupt_scrub_erasure: echo MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T06:02:18.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3665: corrupt_scrub_erasure: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/DIFFERENT 2026-02-17T06:02:18.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3668: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T06:02:18.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:02:18.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:02:18.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:02:18.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:02:18.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:02:20.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:02:20.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T06:02:20.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:02:20.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:02:20.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:02:20.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:02:20.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:02:20.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ1 get-attr hinfo_key 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:02:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:21.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:21.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:21.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:21.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:02:21.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:02:21.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:02:21.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:02:21.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:02:21.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:02:21.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:02:21.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:02:21.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:02:21.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:02:21.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:02:21.031 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:02:21.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:02:21.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:02:21.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:02:21.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:02:21.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:02:21.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:02:21.045 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:21.031+0000 7fb7274ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:21.046 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:21.035+0000 7fb7274ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:21.047 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:21.035+0000 7fb7274ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:21.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:02:21.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:21.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:22.291 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:22.279+0000 7fb7274ba8c0 -1 Falling back to public interface 2026-02-17T06:02:22.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:22.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:22.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:02:22.334 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:22.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:22.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:22.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:23.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:23.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:23.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:02:23.484 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:02:23.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:23.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:23.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:24.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:24.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:24.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:02:24.640 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:02:24.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:24.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:24.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:25.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:25.263+0000 7fb7274ba8c0 -1 osd.0 90 log_to_monitors true 2026-02-17T06:02:25.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:25.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:25.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:02:25.793 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:02:25.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:25.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:25.944 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 94 up_thru 94 down_at 91 last_clean_interval [82,90) [v2:127.0.0.1:6802/2885468545,v1:127.0.0.1:6803/2885468545] [v2:127.0.0.1:6804/2885468545,v1:127.0.0.1:6805/2885468545] exists,up 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:02:25.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:02:25.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:02:25.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:02:25.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:02:25.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:02:25.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:02:25.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:02:25.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:02:25.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:02:25.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:02:25.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:02:25.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:02:25.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:02:26.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:02:26.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:02:26.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:02:26.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:02:26.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:02:26.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:02:26.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:02:26.207 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:26.207 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:02:26.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:02:26.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:26.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:02:26.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=403726925826 2026-02-17T06:02:26.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 403726925826 2026-02-17T06:02:26.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-403726925826' 2026-02-17T06:02:26.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:26.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:02:26.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154757 2026-02-17T06:02:26.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154757 2026-02-17T06:02:26.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-403726925826 1-373662154757' 2026-02-17T06:02:26.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:26.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:02:26.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776142 2026-02-17T06:02:26.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776142 2026-02-17T06:02:26.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-403726925826 1-373662154757 2-292057776142' 2026-02-17T06:02:26.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:26.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-403726925826 2026-02-17T06:02:26.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:26.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:02:26.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-403726925826 2026-02-17T06:02:26.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:26.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=403726925826 2026-02-17T06:02:26.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 403726925826' 2026-02-17T06:02:26.429 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 403726925826 2026-02-17T06:02:26.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:26.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 403726925826 -lt 403726925826 2026-02-17T06:02:26.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:26.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154757 2026-02-17T06:02:26.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:26.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:02:26.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154757 2026-02-17T06:02:26.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:26.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154757 2026-02-17T06:02:26.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154757' 2026-02-17T06:02:26.590 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 373662154757 2026-02-17T06:02:26.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:02:26.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154757 -lt 373662154757 2026-02-17T06:02:26.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:26.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776142 2026-02-17T06:02:26.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:26.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:02:26.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776142 2026-02-17T06:02:26.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:26.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776142 2026-02-17T06:02:26.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776142' 2026-02-17T06:02:26.748 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 292057776142 2026-02-17T06:02:26.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:26.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776141 -lt 292057776142 2026-02-17T06:02:26.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:27.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:02:27.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:28.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776141 -lt 292057776142 2026-02-17T06:02:28.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:29.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:02:29.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:29.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776142 -lt 292057776142 2026-02-17T06:02:29.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:02:29.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:29.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:29.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:02:29.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:02:29.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:02:29.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:02:29.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:02:29.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:02:29.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:02:29.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:02:29.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:02:29.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:02:29.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:29.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3669: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:02:29.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:02:29.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:02:29.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:02:29.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:02:29.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:02:29.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:02:31.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:02:31.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:02:31.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:02:31.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:02:31.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:02:31.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:02:31.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:02:31.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:02:32.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:02:32.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:32.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:02:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:02:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:02:32.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:02:32.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:02:32.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:02:32.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:02:32.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:02:32.246 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:32.235+0000 7f45dbe218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:32.247 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:32.235+0000 7f45dbe218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:32.248 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:32.235+0000 7f45dbe218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:32.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:02:32.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:32.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:32.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:33.484 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:33.471+0000 7f45dbe218c0 -1 Falling back to public interface 2026-02-17T06:02:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:02:33.528 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:33.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:33.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:34.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:02:34.680 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:02:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:34.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:34.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:35.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:35.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:35.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:02:35.831 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:02:35.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:35.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:35.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:02:36.729 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:36.714+0000 7f45dbe218c0 -1 osd.0 95 log_to_monitors true 2026-02-17T06:02:36.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:02:36.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:02:36.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:02:36.985 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:02:36.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:02:36.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 99 up_thru 99 down_at 96 last_clean_interval [94,95) [v2:127.0.0.1:6802/1928235170,v1:127.0.0.1:6803/1928235170] [v2:127.0.0.1:6804/1928235170,v1:127.0.0.1:6805/1928235170] exists,up 3c55f2f2-59bb-463d-8f79-35cc0ab4c248 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:02:37.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:02:37.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:02:37.133 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:02:37.133 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:02:37.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:02:37.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:02:37.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:02:37.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:02:37.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:02:37.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:02:37.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:02:37.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:02:37.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:02:37.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:02:37.393 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:37.393 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:02:37.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:02:37.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:37.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:02:37.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762306 2026-02-17T06:02:37.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762306 2026-02-17T06:02:37.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762306' 2026-02-17T06:02:37.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:37.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:02:37.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154760 2026-02-17T06:02:37.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154760 2026-02-17T06:02:37.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762306 1-373662154760' 2026-02-17T06:02:37.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:37.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:02:37.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776145 2026-02-17T06:02:37.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776145 2026-02-17T06:02:37.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762306 1-373662154760 2-292057776145' 2026-02-17T06:02:37.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:37.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-425201762306 2026-02-17T06:02:37.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:37.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:02:37.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-425201762306 2026-02-17T06:02:37.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:37.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762306 2026-02-17T06:02:37.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 425201762306' 2026-02-17T06:02:37.608 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 425201762306 2026-02-17T06:02:37.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:37.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 403726925826 -lt 425201762306 2026-02-17T06:02:37.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:38.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:02:38.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:38.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762306 2026-02-17T06:02:38.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:38.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154760 2026-02-17T06:02:38.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:38.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:02:38.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154760 2026-02-17T06:02:38.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:38.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154760 2026-02-17T06:02:38.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154760' 2026-02-17T06:02:38.930 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 373662154760 2026-02-17T06:02:38.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:02:39.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154760 -lt 373662154760 2026-02-17T06:02:39.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:39.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776145 2026-02-17T06:02:39.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:39.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:02:39.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776145 2026-02-17T06:02:39.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:39.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776145 2026-02-17T06:02:39.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776145' 2026-02-17T06:02:39.090 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 292057776145 2026-02-17T06:02:39.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:39.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776145 -lt 292057776145 2026-02-17T06:02:39.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:02:39.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:39.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:39.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:02:39.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:02:39.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:02:39.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:02:39.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:02:39.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:02:39.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:02:39.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:02:39.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:02:39.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:02:39.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:02:39.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:02:39.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:02:39.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:02:39.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:02:39.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3670: corrupt_scrub_erasure: rm -f td/osd-scrub-repair/hinfo 2026-02-17T06:02:39.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: get_pg ecpool EOBJ0 2026-02-17T06:02:39.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T06:02:39.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=EOBJ0 2026-02-17T06:02:39.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool EOBJ0 2026-02-17T06:02:39.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:02:39.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: local pg=3.0 2026-02-17T06:02:39.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3678: corrupt_scrub_erasure: pg_scrub 3.0 2026-02-17T06:02:39.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T06:02:39.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T06:02:39.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:02:39.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:02:39.912 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:02:39.912 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:02:39.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:02:39.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:02:39.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:02:40.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:02:40.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:02:40.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:02:40.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:02:40.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:02:40.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:02:40.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:02:40.344 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:40.344 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:02:40.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:02:40.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:40.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:02:40.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762307 2026-02-17T06:02:40.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762307 2026-02-17T06:02:40.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762307' 2026-02-17T06:02:40.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:40.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:02:40.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154762 2026-02-17T06:02:40.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154762 2026-02-17T06:02:40.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762307 1-373662154762' 2026-02-17T06:02:40.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:40.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:02:40.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776147 2026-02-17T06:02:40.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776147 2026-02-17T06:02:40.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762307 1-373662154762 2-292057776147' 2026-02-17T06:02:40.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:40.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-425201762307 2026-02-17T06:02:40.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:40.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:02:40.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-425201762307 2026-02-17T06:02:40.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:40.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762307 2026-02-17T06:02:40.564 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 425201762307 2026-02-17T06:02:40.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 425201762307' 2026-02-17T06:02:40.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762307 2026-02-17T06:02:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:41.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:02:41.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:41.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762307 2026-02-17T06:02:41.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:42.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:02:42.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:43.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762308 -lt 425201762307 2026-02-17T06:02:43.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:43.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154762 2026-02-17T06:02:43.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:43.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:02:43.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154762 2026-02-17T06:02:43.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:43.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154762 2026-02-17T06:02:43.035 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 373662154762 2026-02-17T06:02:43.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154762' 2026-02-17T06:02:43.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:02:43.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154762 -lt 373662154762 2026-02-17T06:02:43.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:43.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776147 2026-02-17T06:02:43.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:02:43.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776147 2026-02-17T06:02:43.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:43.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776147 2026-02-17T06:02:43.190 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 292057776147 2026-02-17T06:02:43.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776147' 2026-02-17T06:02:43.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776147 -lt 292057776147 2026-02-17T06:02:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:02:43.342 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:02:43.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:02:43.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:02:43.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:02:43.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:02:43.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:02:43.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:02:43.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:43.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:02:43.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:43.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T06:02:43.705 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0s0 on osd.1 to scrub 2026-02-17T06:02:43.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:43.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:02:43.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:43.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:02:43.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:02:43.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:43.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:02:43.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:43.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:02:43.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:43.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:02:43.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:43.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:44.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:44.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:44.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:02:44.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:44.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:02:44.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:44.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:02:45.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:45.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:46.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:46.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:46.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:02:46.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:46.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:02:46.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:46.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:02:46.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:46.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:47.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:47.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:47.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:02:47.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:47.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:02:47.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:47.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:02:47.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:02:44.188837+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:47.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:02:47.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3680: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T06:02:47.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T06:02:47.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: test 1 = 1 2026-02-17T06:02:47.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T06:02:47.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T06:02:47.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3686: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T06:02:47.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T06:02:47.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: epoch=99 2026-02-17T06:02:47.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:02:47.387 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:02:47.388 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T06:02:47.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:02:47.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:02:47.406 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:02:47.407 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:02:47.407 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:02:47.407 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T06:02:47.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:02:47.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:02:47.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4311: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:02:47.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:02:47.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4312: corrupt_scrub_erasure: test no = yes 2026-02-17T06:02:47.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4317: corrupt_scrub_erasure: test '' = yes 2026-02-17T06:02:47.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4322: corrupt_scrub_erasure: pg_deep_scrub 3.0 2026-02-17T06:02:47.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T06:02:47.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T06:02:47.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:02:47.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:02:47.433 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:02:47.433 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:02:47.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:02:47.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:02:47.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:02:47.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:02:47.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:02:47.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:02:47.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:02:47.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:02:47.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:02:47.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:02:47.872 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:02:47.872 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:02:47.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:02:47.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:47.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:02:47.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762310 2026-02-17T06:02:47.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762310 2026-02-17T06:02:47.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762310' 2026-02-17T06:02:47.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:47.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:02:48.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154764 2026-02-17T06:02:48.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154764 2026-02-17T06:02:48.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762310 1-373662154764' 2026-02-17T06:02:48.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:02:48.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:02:48.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776149 2026-02-17T06:02:48.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776149 2026-02-17T06:02:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762310 1-373662154764 2-292057776149' 2026-02-17T06:02:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:48.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-425201762310 2026-02-17T06:02:48.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:48.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:02:48.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-425201762310 2026-02-17T06:02:48.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:48.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762310 2026-02-17T06:02:48.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 425201762310' 2026-02-17T06:02:48.092 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 425201762310 2026-02-17T06:02:48.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:48.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762308 -lt 425201762310 2026-02-17T06:02:48.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:02:49.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:02:49.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:02:49.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762310 -lt 425201762310 2026-02-17T06:02:49.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:49.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154764 2026-02-17T06:02:49.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:49.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:02:49.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154764 2026-02-17T06:02:49.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:49.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154764 2026-02-17T06:02:49.410 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 373662154764 2026-02-17T06:02:49.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154764' 2026-02-17T06:02:49.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:02:49.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154764 -lt 373662154764 2026-02-17T06:02:49.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:02:49.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776149 2026-02-17T06:02:49.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:02:49.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:02:49.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776149 2026-02-17T06:02:49.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:02:49.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776149 2026-02-17T06:02:49.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776149' 2026-02-17T06:02:49.574 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 292057776149 2026-02-17T06:02:49.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:02:49.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776149 -lt 292057776149 2026-02-17T06:02:49.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:02:49.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:02:49.735 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:02:49.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:02:49.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:02:49.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:02:49.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:02:49.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:02:49.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-17T06:02:49.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:02:49.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:02:49.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:02:49.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:49.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:49.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:49.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:49.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:49.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:49.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T06:02:50.101 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0s0 on osd.1 to deep-scrub 2026-02-17T06:02:50.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T06:00:37.944853+0000 last_deep_scrub_stamp 2026-02-17T06:02:50.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:02:50.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:50.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T06:02:50.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:02:50.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:50.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:50.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:50.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:50.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:50.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:50.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:50.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:51.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:51.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:51.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:51.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:51.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:51.264 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:51.264 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:51.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:51.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:52.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:52.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:52.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:52.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:52.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:52.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:52.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:52.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:52.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:53.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:53.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:53.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:53.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:53.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:53.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:53.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:53.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:53.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:54.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:54.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:54.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:54.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:54.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:54.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:54.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:54.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:54.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:55.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:55.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:55.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:55.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:55.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:55.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:55.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:56.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:00:37.944853+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:56.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:02:57.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:02:57.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:02:57.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:02:57.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:02:57.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:02:57.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:02:57.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:02:57.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:02:50.276658+0000 '>' 2026-02-17T06:00:37.944853+0000 2026-02-17T06:02:57.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:02:57.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4324: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T06:02:57.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T06:02:57.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: test 1 = 1 2026-02-17T06:02:57.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T06:02:57.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T06:02:57.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4330: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T06:02:57.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T06:02:57.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: epoch=99 2026-02-17T06:02:57.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4334: corrupt_scrub_erasure: '[' false = true ']' 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4991: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4991: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:02:57.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4991: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:02:57.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:02:57.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:02:57.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5725: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:02:57.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:02:57.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5726: corrupt_scrub_erasure: test no = yes 2026-02-17T06:02:57.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5737: corrupt_scrub_erasure: test '' = yes 2026-02-17T06:02:57.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5742: corrupt_scrub_erasure: ceph osd pool rm ecpool ecpool --yes-i-really-really-mean-it 2026-02-17T06:02:57.486 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' removed 2026-02-17T06:02:57.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:02:57.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:02:57.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:02:57.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:02:57.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:02:57.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:02:57.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:02:57.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:02:57.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:02:57.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:02:57.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:02:57.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:02:57.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:02:57.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:02:57.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:02:57.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:02:57.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:02:57.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:02:57.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:02:57.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:02:57.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:02:57.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:02:57.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:02:57.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:02:57.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:02:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:02:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:02:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:02:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:02:57.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:02:57.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:02:57.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:02:57.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:02:57.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:02:57.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:02:57.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:02:57.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:02:57.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:02:57.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:02:57.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:02:57.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:02:57.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:02:57.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:02:57.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:02:57.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:02:57.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:02:57.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:02:57.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:02:57.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:02:57.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:02:57.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:02:57.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:02:57.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:02:57.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_erasure_overwrites td/osd-scrub-repair 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5750: TEST_corrupt_scrub_erasure_overwrites: '[' true = true ']' 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5751: TEST_corrupt_scrub_erasure_overwrites: corrupt_scrub_erasure td/osd-scrub-repair true 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3596: corrupt_scrub_erasure: local dir=td/osd-scrub-repair 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3597: corrupt_scrub_erasure: local allow_overwrites=true 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3598: corrupt_scrub_erasure: local poolname=ecpool 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3599: corrupt_scrub_erasure: local total_objs=7 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3601: corrupt_scrub_erasure: run_mon td/osd-scrub-repair a 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:02:57.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:02:57.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:02:57.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:02:57.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:02:57.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:02:57.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:02:57.684 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:02:57.684 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.684 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:57.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:02:57.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:02:57.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:02:57.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:02:57.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:02:57.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:02:57.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:02:57.716 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:02:57.716 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:02:57.716 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:02:57.716 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:02:57.716 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.716 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.716 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:02:57.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:02:57.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:02:57.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:02:57.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:02:57.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:02:57.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:02:57.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:02:57.772 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:02:57.772 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:02:57.772 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:02:57.773 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:02:57.773 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.773 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.774 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:02:57.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:02:57.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:02:57.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3602: corrupt_scrub_erasure: run_mgr td/osd-scrub-repair x 2026-02-17T06:02:57.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:02:57.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:02:57.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:02:57.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:02:57.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:02:57.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:02:57.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:02:57.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:02:57.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:02:57.920 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:02:57.920 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.920 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:57.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:02:57.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:02:57.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: seq 0 2 2026-02-17T06:02:57.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T06:02:57.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 0 2026-02-17T06:02:57.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:02:57.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:02:57.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:02:57.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:02:57.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:02:57.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:02:57.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:02:57.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:02:57.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:02:57.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:02:57.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:02:57.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:02:57.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:02:57.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:02:57.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 1bf6acc4-3753-42db-b389-e73235328388' 2026-02-17T06:02:57.948 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:02:57.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:02:57.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCRBJRpdjR8OBAAph8poGDbk6eQgDXrP/669Q== 2026-02-17T06:02:57.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCRBJRpdjR8OBAAph8poGDbk6eQgDXrP/669Q=="}' 2026-02-17T06:02:57.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1bf6acc4-3753-42db-b389-e73235328388 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:02:58.047 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:02:58.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:02:58.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCRBJRpdjR8OBAAph8poGDbk6eQgDXrP/669Q== --osd-uuid 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:02:58.071 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:58.058+0000 7f73a89c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:58.073 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:58.058+0000 7f73a89c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:58.074 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:58.062+0000 7f73a89c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:02:58.074 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:58.062+0000 7f73a89c18c0 -1 bdev(0x562baa7afc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:02:58.074 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:02:58.062+0000 7f73a89c18c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:03:00.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:03:00.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:03:00.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:03:00.640 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:03:00.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:03:00.831 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:03:00.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:03:00.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:03:00.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:03:00.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:03:00.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:03:00.854 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:00.838+0000 7fc533b398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:00.855 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:00.842+0000 7fc533b398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:00.856 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:00.842+0000 7fc533b398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:03:00.981 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:03:00.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:00.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:01.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:02.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:02.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:02.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:03:02.134 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:03:02.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:02.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:02.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:02.359 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:02.346+0000 7fc533b398c0 -1 Falling back to public interface 2026-02-17T06:03:03.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:03.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:03.290 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:03:03.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:03:03.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:03.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:03.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:04.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:04.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:04.445 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:03:04.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:03:04.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:04.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:04.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:05.334 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:05.322+0000 7fc533b398c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:03:05.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:05.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:05.598 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:03:05.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:03:05.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:05.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:05.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:06.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:06.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:06.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:03:06.751 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:03:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:06.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:06.900 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2991619778,v1:127.0.0.1:6803/2991619778] [v2:127.0.0.1:6804/2991619778,v1:127.0.0.1:6805/2991619778] exists,up 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 1 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:03:06.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:03:06.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:03:06.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:03:06.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:03:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:03:06.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:03:06.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:03:06.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 720800fd-904c-4c90-946c-3f8dcccbe5f5' 2026-02-17T06:03:06.906 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:03:06.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:03:06.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCaBJRpQEAoNhAAXWh5vQvVcr/Dv+2mmRT23w== 2026-02-17T06:03:06.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCaBJRpQEAoNhAAXWh5vQvVcr/Dv+2mmRT23w=="}' 2026-02-17T06:03:06.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 720800fd-904c-4c90-946c-3f8dcccbe5f5 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:03:07.064 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:03:07.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:03:07.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCaBJRpQEAoNhAAXWh5vQvVcr/Dv+2mmRT23w== --osd-uuid 720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:03:07.090 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:07.078+0000 7ff9dd4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:07.092 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:07.078+0000 7ff9dd4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:07.093 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:07.078+0000 7ff9dd4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:07.093 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:07.078+0000 7ff9dd4538c0 -1 bdev(0x55ebe8b63c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:03:07.093 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:07.078+0000 7ff9dd4538c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:03:09.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:03:09.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:03:09.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:03:09.392 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:03:09.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:03:09.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:03:09.584 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:03:09.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:03:09.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:03:09.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:03:09.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:03:09.607 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:09.594+0000 7fe6afa3b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:09.609 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:09.594+0000 7fe6afa3b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:09.610 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:09.598+0000 7fe6afa3b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:09.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:09.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:10.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:10.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:10.888 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:03:10.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:03:10.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:10.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:11.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:11.087 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:11.074+0000 7fe6afa3b8c0 -1 Falling back to public interface 2026-02-17T06:03:12.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:12.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:12.042 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:03:12.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:03:12.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:12.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:12.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:13.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:13.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:13.193 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:03:13.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:03:13.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:13.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:13.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:14.049 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:14.034+0000 7fe6afa3b8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:03:14.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:14.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:14.342 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:03:14.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:03:14.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:14.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:14.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:15.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:15.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:15.492 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:03:15.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:03:15.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:15.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:15.639 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2434542093,v1:127.0.0.1:6811/2434542093] [v2:127.0.0.1:6812/2434542093,v1:127.0.0.1:6813/2434542093] exists,up 720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 2 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:03:15.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:03:15.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:03:15.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:03:15.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:03:15.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=09b37cb8-395c-4930-8d6e-b3db4e339e9c 2026-02-17T06:03:15.645 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 09b37cb8-395c-4930-8d6e-b3db4e339e9c 2026-02-17T06:03:15.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 09b37cb8-395c-4930-8d6e-b3db4e339e9c' 2026-02-17T06:03:15.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:03:15.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCjBJRpSwOQJhAAl4ijkLgvpooj24kRShayUg== 2026-02-17T06:03:15.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCjBJRpSwOQJhAAl4ijkLgvpooj24kRShayUg=="}' 2026-02-17T06:03:15.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 09b37cb8-395c-4930-8d6e-b3db4e339e9c -i td/osd-scrub-repair/2/new.json 2026-02-17T06:03:15.810 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:03:15.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T06:03:15.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCjBJRpSwOQJhAAl4ijkLgvpooj24kRShayUg== --osd-uuid 09b37cb8-395c-4930-8d6e-b3db4e339e9c 2026-02-17T06:03:15.835 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:15.822+0000 7fd488c118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:15.837 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:15.822+0000 7fd488c118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:15.838 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:15.826+0000 7fd488c118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:15.839 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:15.826+0000 7fd488c118c0 -1 bdev(0x55f1a05b7c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:03:15.839 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:15.826+0000 7fd488c118c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T06:03:18.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T06:03:18.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:03:18.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:03:18.104 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:03:18.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:03:18.298 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:03:18.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:03:18.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:03:18.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:03:18.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:03:18.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:03:18.321 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:18.306+0000 7f680d4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:18.323 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:18.310+0000 7f680d4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:18.324 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:18.310+0000 7f680d4538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:18.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:03:18.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:03:18.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:03:18.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:03:18.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:03:18.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:18.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:03:18.452 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:03:18.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:18.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:03:18.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:19.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:19.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:19.603 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:03:19.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:03:19.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:19.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:03:19.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:19.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:19.802+0000 7f680d4538c0 -1 Falling back to public interface 2026-02-17T06:03:20.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:20.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:20.753 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:03:20.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:03:20.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:20.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:03:20.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:21.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:21.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:21.902 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:03:21.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:03:21.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:21.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:03:22.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:23.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:23.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:23.056 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:03:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:03:23.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:23.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:03:23.063 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:23.050+0000 7f680d4538c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:03:23.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:24.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:24.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:24.212 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:03:24.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:03:24.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:24.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:03:24.357 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2019693425,v1:127.0.0.1:6819/2019693425] [v2:127.0.0.1:6820/2019693425,v1:127.0.0.1:6821/2019693425] exists,up 09b37cb8-395c-4930-8d6e-b3db4e339e9c 2026-02-17T06:03:24.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:03:24.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:03:24.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:03:24.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3606: corrupt_scrub_erasure: create_rbd_pool 2026-02-17T06:03:24.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:03:24.497 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:03:24.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:03:24.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:03:24.720 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:03:24.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:03:25.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:03:26.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3607: corrupt_scrub_erasure: create_pool foo 1 2026-02-17T06:03:26.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T06:03:26.211 INFO:tasks.workunit.client.0.trial074.stderr:pool 'foo' created 2026-02-17T06:03:26.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:03:27.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3609: corrupt_scrub_erasure: create_ec_pool ecpool true k=2 m=1 stripe_unit=2K --force 2026-02-17T06:03:27.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T06:03:27.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T06:03:27.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T06:03:27.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T06:03:27.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 stripe_unit=2K --force 2026-02-17T06:03:27.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T06:03:27.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T06:03:27.681 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T06:03:27.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:03:28.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T06:03:28.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T06:03:28.888 INFO:tasks.workunit.client.0.trial074.stderr:set pool 3 allow_ec_overwrites to true 2026-02-17T06:03:28.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T06:03:28.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:03:28.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:03:28.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:03:28.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:03:28.903 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:03:28.903 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:03:28.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:03:28.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:03:28.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:03:29.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:03:29.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:03:29.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:03:29.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:03:29.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:03:29.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:03:29.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:03:29.173 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:03:29.173 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:03:29.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:03:29.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:29.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:03:29.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T06:03:29.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T06:03:29.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T06:03:29.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:29.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:03:29.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T06:03:29.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T06:03:29.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705668' 2026-02-17T06:03:29.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:29.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:03:29.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-17T06:03:29.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-17T06:03:29.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705668 2-60129542147' 2026-02-17T06:03:29.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:29.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T06:03:29.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:29.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:03:29.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T06:03:29.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:29.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T06:03:29.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T06:03:29.393 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836486 2026-02-17T06:03:29.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:29.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T06:03:29.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:03:30.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:03:30.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:30.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T06:03:30.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:30.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T06:03:30.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:30.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:03:30.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T06:03:30.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:30.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T06:03:30.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T06:03:30.696 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705668 2026-02-17T06:03:30.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:03:30.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T06:03:30.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:30.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-17T06:03:30.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:30.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:03:30.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-17T06:03:30.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:30.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-17T06:03:30.863 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542147 2026-02-17T06:03:30.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T06:03:30.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:03:31.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-17T06:03:31.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:03:31.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:31.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:31.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:03:31.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:03:31.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:03:31.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:03:31.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:03:31.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:03:31.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:03:31.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:03:31.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:03:31.340 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:03:31.341 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:31.341 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:31.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:03:31.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:03:31.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:03:31.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3610: corrupt_scrub_erasure: wait_for_clean 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:03:31.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:03:31.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:03:31.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:03:31.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:03:31.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:03:31.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:03:31.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:03:31.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:03:31.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:03:31.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:03:31.788 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:03:31.789 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:03:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:03:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:31.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:03:31.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T06:03:31.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T06:03:31.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T06:03:31.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:31.862 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:03:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T06:03:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T06:03:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670' 2026-02-17T06:03:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:31.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:03:32.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T06:03:32.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T06:03:32.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670 2-60129542148' 2026-02-17T06:03:32.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:32.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T06:03:32.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:32.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:03:32.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T06:03:32.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:32.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T06:03:32.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T06:03:32.013 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836488 2026-02-17T06:03:32.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:32.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-17T06:03:32.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:03:33.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:03:33.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:33.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T06:03:33.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:33.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T06:03:33.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:33.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:03:33.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T06:03:33.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:33.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T06:03:33.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T06:03:33.318 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705670 2026-02-17T06:03:33.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:03:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T06:03:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:33.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T06:03:33.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:33.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:03:33.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T06:03:33.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:33.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T06:03:33.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T06:03:33.479 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542148 2026-02-17T06:03:33.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:03:33.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T06:03:33.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:03:33.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:33.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:33.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:03:33.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:03:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:03:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:03:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:03:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:03:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:03:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:03:33.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:03:33.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:03:33.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:33.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:34.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:03:34.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:03:34.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:03:34.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: seq 1 7 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ1 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ1 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ1 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:03:34.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:03:34.344 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:03:34.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:03:34.549 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:03:34.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:03:34.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:03:34.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:03:34.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 1 % 2 2026-02-17T06:03:34.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3621: corrupt_scrub_erasure: local payload=UVWXYZZZ 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3622: corrupt_scrub_erasure: echo UVWXYZZZ 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3623: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:03:34.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:03:34.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:03:34.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:03:35.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:03:35.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:35.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:03:35.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:03:35.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:03:35.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:03:35.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:03:35.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:37.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:03:37.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:03:37.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:03:37.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:03:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:37.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:03:37.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:03:37.091 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:03:37.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:03:37.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:03:37.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:03:37.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:03:37.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:03:37.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:03:37.109 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:37.094+0000 7f158b5028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:37.110 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:37.098+0000 7f158b5028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:37.111 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:37.098+0000 7f158b5028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:03:37.242 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:03:37.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:37.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:38.351 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:38.338+0000 7f158b5028c0 -1 Falling back to public interface 2026-02-17T06:03:38.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:38.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:38.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:03:38.391 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:03:38.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:38.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:38.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:39.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:39.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:39.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:03:39.541 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:03:39.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:39.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:39.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:40.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:40.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:40.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:03:40.692 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:03:40.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:40.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:40.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:41.332 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:41.318+0000 7f158b5028c0 -1 osd.1 33 log_to_monitors true 2026-02-17T06:03:41.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:41.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:41.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:03:41.844 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:03:41.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:41.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [9,33) [v2:127.0.0.1:6810/3234120541,v1:127.0.0.1:6811/3234120541] [v2:127.0.0.1:6812/3234120541,v1:127.0.0.1:6813/3234120541] exists,up 720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:03:41.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:03:41.994 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:03:41.994 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:03:41.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:03:41.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:03:41.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:03:42.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:03:42.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:03:42.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:03:42.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:03:42.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:03:42.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:03:42.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:03:42.258 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:03:42.258 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:03:42.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:03:42.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:42.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:03:42.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T06:03:42.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T06:03:42.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T06:03:42.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:42.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:03:42.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789954 2026-02-17T06:03:42.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789954 2026-02-17T06:03:42.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-158913789954' 2026-02-17T06:03:42.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:42.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:03:42.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542151 2026-02-17T06:03:42.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542151 2026-02-17T06:03:42.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-158913789954 2-60129542151' 2026-02-17T06:03:42.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:42.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T06:03:42.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:42.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:03:42.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T06:03:42.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:42.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T06:03:42.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T06:03:42.477 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836491 2026-02-17T06:03:42.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:42.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T06:03:42.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:03:43.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:03:43.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:43.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T06:03:43.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:03:44.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:03:44.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:44.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T06:03:44.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:44.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789954 2026-02-17T06:03:44.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:44.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:03:44.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789954 2026-02-17T06:03:44.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:44.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789954 2026-02-17T06:03:44.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789954' 2026-02-17T06:03:44.954 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 158913789954 2026-02-17T06:03:44.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:03:45.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789954 -lt 158913789954 2026-02-17T06:03:45.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:45.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542151 2026-02-17T06:03:45.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:45.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:03:45.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542151 2026-02-17T06:03:45.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:45.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542151 2026-02-17T06:03:45.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-17T06:03:45.109 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542151 2026-02-17T06:03:45.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:03:45.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542151 -lt 60129542151 2026-02-17T06:03:45.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:03:45.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:45.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:45.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:03:45.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:03:45.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:03:45.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:03:45.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:03:45.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:03:45.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:03:45.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:03:45.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:03:45.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:03:45.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:45.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:45.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ2 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ2 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ2 2026-02-17T06:03:45.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:03:45.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:03:45.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:03:45.986 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:03:45.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:03:46.190 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:03:46.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:03:46.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:03:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:03:46.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 2 % 2 2026-02-17T06:03:46.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T06:03:46.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:03:46.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3628: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=1 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:2048 bytes (2.0 kB, 2.0 KiB) copied, 9.3296e-05 s, 22.0 MB/s 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3629: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:03:46.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:03:46.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:03:46.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:03:46.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:03:46.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:03:46.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:03:47.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:03:48.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:03:48.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:03:48.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:03:48.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:03:48.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:03:48.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:03:48.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:03:48.729 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:03:48.729 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:03:48.729 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:03:48.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:03:48.731 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:03:48.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:03:48.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:03:48.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:03:48.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:03:48.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:03:48.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:03:48.754 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:48.742+0000 7f9b194238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:48.755 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:48.742+0000 7f9b194238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:48.756 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:48.742+0000 7f9b194238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:03:48.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:48.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:49.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:50.007 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:49.994+0000 7f9b194238c0 -1 Falling back to public interface 2026-02-17T06:03:50.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:50.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:50.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:03:50.036 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:03:50.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:50.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:50.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:51.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:51.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:51.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:03:51.191 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:03:51.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:51.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:51.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:52.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:52.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:52.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:03:52.344 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:03:52.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:52.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:52.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:52.985 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:52.974+0000 7f9b194238c0 -1 osd.0 40 log_to_monitors true 2026-02-17T06:03:53.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:53.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:53.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:03:53.496 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:03:53.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:53.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:03:53.646 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 44 up_thru 44 down_at 41 last_clean_interval [5,40) [v2:127.0.0.1:6802/3388266943,v1:127.0.0.1:6803/3388266943] [v2:127.0.0.1:6804/3388266943,v1:127.0.0.1:6805/3388266943] exists,up 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:03:53.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:03:53.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:03:53.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:03:53.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:03:53.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:03:53.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:03:53.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:03:53.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:03:53.648 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:03:53.648 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:03:53.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:03:53.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:03:53.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:03:53.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:03:53.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:03:53.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:03:53.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:03:53.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:03:53.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:03:53.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:03:53.907 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:03:53.908 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:03:53.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:03:53.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:53.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:03:53.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T06:03:53.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T06:03:53.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561026' 2026-02-17T06:03:53.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:53.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:03:54.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789957 2026-02-17T06:03:54.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789957 2026-02-17T06:03:54.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561026 1-158913789957' 2026-02-17T06:03:54.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:03:54.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:03:54.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542154 2026-02-17T06:03:54.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542154 2026-02-17T06:03:54.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561026 1-158913789957 2-60129542154' 2026-02-17T06:03:54.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:54.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-188978561026 2026-02-17T06:03:54.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:54.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:03:54.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-188978561026 2026-02-17T06:03:54.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:54.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T06:03:54.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 188978561026' 2026-02-17T06:03:54.125 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 188978561026 2026-02-17T06:03:54.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:03:54.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T06:03:54.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:54.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789957 2026-02-17T06:03:54.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:54.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:03:54.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789957 2026-02-17T06:03:54.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:54.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789957 2026-02-17T06:03:54.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789957' 2026-02-17T06:03:54.288 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 158913789957 2026-02-17T06:03:54.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:03:54.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789957 -lt 158913789957 2026-02-17T06:03:54.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:03:54.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542154 2026-02-17T06:03:54.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:03:54.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:03:54.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542154 2026-02-17T06:03:54.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:03:54.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542154 2026-02-17T06:03:54.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542154' 2026-02-17T06:03:54.454 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542154 2026-02-17T06:03:54.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:03:54.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542154 -lt 60129542154 2026-02-17T06:03:54.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:03:54.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:54.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:54.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:03:54.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:03:54.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:03:54.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:03:54.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:03:54.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:03:54.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:03:54.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:03:54.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:03:54.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:03:54.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:03:54.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ3 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ3 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ3 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:03:55.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:03:55.318 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:03:55.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:03:55.523 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:03:55.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:03:55.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:03:55.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:03:55.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 3 % 2 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3634: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:03:55.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:03:55.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:03:56.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:03:56.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T06:03:56.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:03:56.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:03:56.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:03:56.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:03:56.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:03:56.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ3 remove 2026-02-17T06:03:57.495 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#3:b197b25d:::EOBJ3:head# 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:03:58.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:03:58.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:03:58.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:03:58.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:03:58.033 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:03:58.033 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:03:58.033 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:03:58.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:03:58.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:03:58.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:03:58.035 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:03:58.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:03:58.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:03:58.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:03:58.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:03:58.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:03:58.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:03:58.059 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:58.046+0000 7f69916a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:58.060 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:58.046+0000 7f69916a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:58.061 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:58.050+0000 7f69916a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:03:58.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:58.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:58.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:03:59.311 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:03:59.298+0000 7f69916a08c0 -1 Falling back to public interface 2026-02-17T06:03:59.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:03:59.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:03:59.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:03:59.341 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:03:59.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:03:59.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:03:59.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:00.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:00.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:00.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:04:00.491 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:04:00.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:00.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:00.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:01.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:01.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:01.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:04:01.640 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:04:01.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:01.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:01.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:02.274 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:02.262+0000 7f69916a08c0 -1 osd.1 48 log_to_monitors true 2026-02-17T06:04:02.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:02.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:02.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:04:02.787 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:04:02.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:02.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:02.936 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 52 up_thru 52 down_at 49 last_clean_interval [37,48) [v2:127.0.0.1:6810/4197864399,v1:127.0.0.1:6811/4197864399] [v2:127.0.0.1:6812/4197864399,v1:127.0.0.1:6813/4197864399] exists,up 720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:04:02.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:04:02.938 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:04:02.938 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:04:02.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:04:02.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:04:02.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:04:03.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:04:03.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:04:03.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:04:03.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:04:03.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:04:03.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:04:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:04:03.203 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:03.203 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:04:03.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:04:03.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:03.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:04:03.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561029 2026-02-17T06:04:03.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561029 2026-02-17T06:04:03.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561029' 2026-02-17T06:04:03.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:03.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:04:03.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=223338299394 2026-02-17T06:04:03.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 223338299394 2026-02-17T06:04:03.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561029 1-223338299394' 2026-02-17T06:04:03.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:03.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:04:03.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542157 2026-02-17T06:04:03.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542157 2026-02-17T06:04:03.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561029 1-223338299394 2-60129542157' 2026-02-17T06:04:03.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:03.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-188978561029 2026-02-17T06:04:03.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:03.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:04:03.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-188978561029 2026-02-17T06:04:03.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:03.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561029 2026-02-17T06:04:03.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 188978561029' 2026-02-17T06:04:03.419 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 188978561029 2026-02-17T06:04:03.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:03.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561027 -lt 188978561029 2026-02-17T06:04:03.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:04.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:04:04.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:04.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 188978561029 2026-02-17T06:04:04.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:04.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-223338299394 2026-02-17T06:04:04.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:04.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:04:04.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-223338299394 2026-02-17T06:04:04.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:04.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=223338299394 2026-02-17T06:04:04.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 223338299394' 2026-02-17T06:04:04.739 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 223338299394 2026-02-17T06:04:04.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:04.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 223338299394 -lt 223338299394 2026-02-17T06:04:04.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:04.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542157 2026-02-17T06:04:04.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:04.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:04:04.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542157 2026-02-17T06:04:04.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542157 2026-02-17T06:04:04.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-17T06:04:04.904 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542157 2026-02-17T06:04:04.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:04:05.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542157 -lt 60129542157 2026-02-17T06:04:05.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:04:05.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:05.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:05.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:04:05.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:04:05.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:04:05.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:04:05.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:04:05.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:04:05.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:04:05.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:04:05.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:04:05.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:04:05.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:05.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:05.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ4 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ4 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ4 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:04:05.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:04:05.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:04:05.765 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:04:05.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:04:05.969 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:04:05.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:04:05.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:04:05.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:04:06.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 4 % 2 2026-02-17T06:04:06.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T06:04:06.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:04:06.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3638: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key1-EOBJ4 val1-EOBJ4 2026-02-17T06:04:06.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3639: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key2-EOBJ4 val2-EOBJ4 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3642: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3643: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:04:06.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:04:06.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:04:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:04:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:04:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:04:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:04:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:04:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:04:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:04:07.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T06:04:08.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:04:08.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:04:08.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:04:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:04:08.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:04:08.586 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:08.574+0000 7f321834d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:08.588 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:08.574+0000 7f321834d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:08.589 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:08.574+0000 7f321834d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:08.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:08.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:04:09.867 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:09.870 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:09.858+0000 7f321834d8c0 -1 Falling back to public interface 2026-02-17T06:04:10.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:11.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:11.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:11.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:04:11.024 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:04:11.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:11.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:11.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:12.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:12.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:12.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:04:12.176 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:04:12.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:12.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:12.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:12.825 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:12.814+0000 7f321834d8c0 -1 osd.0 55 log_to_monitors true 2026-02-17T06:04:13.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:13.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:13.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:04:13.330 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:04:13.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:13.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:13.480 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [44,55) [v2:127.0.0.1:6802/3542673669,v1:127.0.0.1:6803/3542673669] [v2:127.0.0.1:6804/3542673669,v1:127.0.0.1:6805/3542673669] exists,up 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:04:13.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:04:13.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:04:13.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:04:13.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:04:13.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:04:13.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:04:13.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:04:13.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:04:13.482 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:04:13.482 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:04:13.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:04:13.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:04:13.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:04:13.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:04:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:04:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:04:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:04:13.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:04:13.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:04:13.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:04:13.749 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:13.749 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:04:13.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:04:13.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:13.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:04:13.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=253403070466 2026-02-17T06:04:13.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 253403070466 2026-02-17T06:04:13.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070466' 2026-02-17T06:04:13.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:13.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:04:13.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=223338299397 2026-02-17T06:04:13.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 223338299397 2026-02-17T06:04:13.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070466 1-223338299397' 2026-02-17T06:04:13.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:13.895 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:04:13.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542160 2026-02-17T06:04:13.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542160 2026-02-17T06:04:13.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070466 1-223338299397 2-60129542160' 2026-02-17T06:04:13.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:13.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-253403070466 2026-02-17T06:04:13.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:13.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:04:13.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-253403070466 2026-02-17T06:04:13.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:13.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=253403070466 2026-02-17T06:04:13.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 253403070466' 2026-02-17T06:04:13.970 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 253403070466 2026-02-17T06:04:13.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 253403070466 2026-02-17T06:04:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:15.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:04:15.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:15.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070466 -lt 253403070466 2026-02-17T06:04:15.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:15.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-223338299397 2026-02-17T06:04:15.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:15.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:04:15.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-223338299397 2026-02-17T06:04:15.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:15.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=223338299397 2026-02-17T06:04:15.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 223338299397' 2026-02-17T06:04:15.293 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 223338299397 2026-02-17T06:04:15.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:15.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 223338299397 -lt 223338299397 2026-02-17T06:04:15.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:15.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542160 2026-02-17T06:04:15.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:15.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:04:15.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542160 2026-02-17T06:04:15.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:15.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542160 2026-02-17T06:04:15.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542160' 2026-02-17T06:04:15.457 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 60129542160 2026-02-17T06:04:15.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:04:15.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542160 -lt 60129542160 2026-02-17T06:04:15.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:04:15.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:15.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:15.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:04:15.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:04:15.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:04:15.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:04:15.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:04:15.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:04:15.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:04:15.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:04:15.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:04:15.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:04:15.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:15.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:16.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:04:16.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:04:16.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3644: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:04:16.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:04:16.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:04:16.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:04:16.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:04:17.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T06:04:18.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T06:04:18.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:04:18.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:04:18.630 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:04:18.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:04:18.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:04:18.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:04:18.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:04:18.633 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:04:18.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:04:18.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:04:18.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:04:18.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:04:18.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:04:18.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:04:18.653 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:18.642+0000 7fa9fb29f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:18.655 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:18.642+0000 7fa9fb29f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:18.656 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:18.642+0000 7fa9fb29f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:18.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:18.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:18.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:19.923 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:19.910+0000 7fa9fb29f8c0 -1 Falling back to public interface 2026-02-17T06:04:19.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:19.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:19.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:04:19.937 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:19.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:19.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:20.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:21.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:21.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:21.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:04:21.088 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:04:21.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:21.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:21.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:22.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:22.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:22.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:04:22.240 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:04:22.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:22.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:22.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:22.965 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:22.954+0000 7fa9fb29f8c0 -1 osd.2 60 log_to_monitors true 2026-02-17T06:04:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:23.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:04:23.394 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:04:23.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:23.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:23.544 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 64 up_thru 64 down_at 61 last_clean_interval [14,60) [v2:127.0.0.1:6818/3964950097,v1:127.0.0.1:6819/3964950097] [v2:127.0.0.1:6820/3964950097,v1:127.0.0.1:6821/3964950097] exists,up 09b37cb8-395c-4930-8d6e-b3db4e339e9c 2026-02-17T06:04:23.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:04:23.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:04:23.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:04:23.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:04:23.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:04:23.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:04:23.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:04:23.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:04:23.546 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:04:23.547 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:04:23.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:04:23.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:04:23.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:04:23.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:04:23.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:04:23.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:04:23.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:04:23.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:04:23.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:04:23.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:04:23.810 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:23.810 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:04:23.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:04:23.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:23.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:04:23.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=253403070469 2026-02-17T06:04:23.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 253403070469 2026-02-17T06:04:23.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070469' 2026-02-17T06:04:23.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:23.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:04:23.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=223338299400 2026-02-17T06:04:23.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 223338299400 2026-02-17T06:04:23.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070469 1-223338299400' 2026-02-17T06:04:23.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:23.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:04:24.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906946 2026-02-17T06:04:24.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906946 2026-02-17T06:04:24.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070469 1-223338299400 2-274877906946' 2026-02-17T06:04:24.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:24.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-253403070469 2026-02-17T06:04:24.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:24.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:04:24.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-253403070469 2026-02-17T06:04:24.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:24.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=253403070469 2026-02-17T06:04:24.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 253403070469' 2026-02-17T06:04:24.031 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 253403070469 2026-02-17T06:04:24.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:24.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070467 -lt 253403070469 2026-02-17T06:04:24.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:25.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:04:25.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:25.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070469 -lt 253403070469 2026-02-17T06:04:25.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:25.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-223338299400 2026-02-17T06:04:25.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:25.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:04:25.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-223338299400 2026-02-17T06:04:25.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:25.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=223338299400 2026-02-17T06:04:25.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 223338299400' 2026-02-17T06:04:25.338 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 223338299400 2026-02-17T06:04:25.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:25.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 223338299400 -lt 223338299400 2026-02-17T06:04:25.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:25.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-274877906946 2026-02-17T06:04:25.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:25.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:04:25.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-274877906946 2026-02-17T06:04:25.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:25.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906946 2026-02-17T06:04:25.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 274877906946' 2026-02-17T06:04:25.494 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 274877906946 2026-02-17T06:04:25.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:04:25.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906946 -lt 274877906946 2026-02-17T06:04:25.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:04:25.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:25.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:04:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:04:25.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:04:25.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:04:25.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:25.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:04:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:04:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:04:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3645: corrupt_scrub_erasure: echo -n val3-EOBJ4 2026-02-17T06:04:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3646: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:04:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:04:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:04:26.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:04:26.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:04:26.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:04:27.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T06:04:28.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:04:28.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:04:28.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:04:28.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:04:28.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:04:28.649 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:04:28.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:04:28.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:04:28.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:04:28.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:04:28.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:04:28.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:04:28.668 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:28.654+0000 7f26509fa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:28.669 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:28.658+0000 7f26509fa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:28.670 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:28.658+0000 7f26509fa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:28.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:04:28.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:04:28.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:04:28.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:04:28.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:04:28.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:28.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:04:28.802 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:04:28.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:28.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:28.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:29.930 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:29.918+0000 7f26509fa8c0 -1 Falling back to public interface 2026-02-17T06:04:29.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:29.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:29.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:04:29.956 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:29.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:29.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:30.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:31.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:31.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:31.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:04:31.108 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:04:31.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:31.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:31.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:32.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:32.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:32.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:04:32.256 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:04:32.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:32.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:32.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:32.884 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:32.870+0000 7f26509fa8c0 -1 osd.2 65 log_to_monitors true 2026-02-17T06:04:33.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:33.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:33.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:04:33.405 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:04:33.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:33.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:04:33.555 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 69 up_thru 69 down_at 66 last_clean_interval [64,65) [v2:127.0.0.1:6818/741437991,v1:127.0.0.1:6819/741437991] [v2:127.0.0.1:6820/741437991,v1:127.0.0.1:6821/741437991] exists,up 09b37cb8-395c-4930-8d6e-b3db4e339e9c 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:04:33.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:04:33.557 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:04:33.557 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:04:33.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:04:33.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:04:33.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:04:33.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:04:33.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:04:33.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:04:33.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:04:33.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:04:33.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:04:33.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:04:33.815 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:33.815 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:04:33.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:04:33.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:33.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:04:33.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=253403070472 2026-02-17T06:04:33.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 253403070472 2026-02-17T06:04:33.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070472' 2026-02-17T06:04:33.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:33.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:04:33.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=223338299403 2026-02-17T06:04:33.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 223338299403 2026-02-17T06:04:33.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070472 1-223338299403' 2026-02-17T06:04:33.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:33.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:04:34.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743426 2026-02-17T06:04:34.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743426 2026-02-17T06:04:34.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070472 1-223338299403 2-296352743426' 2026-02-17T06:04:34.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:34.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-253403070472 2026-02-17T06:04:34.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:34.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:04:34.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-253403070472 2026-02-17T06:04:34.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:34.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=253403070472 2026-02-17T06:04:34.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 253403070472' 2026-02-17T06:04:34.036 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 253403070472 2026-02-17T06:04:34.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:34.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070470 -lt 253403070472 2026-02-17T06:04:34.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:35.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:04:35.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:35.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070472 -lt 253403070472 2026-02-17T06:04:35.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:35.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-223338299403 2026-02-17T06:04:35.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:35.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:04:35.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-223338299403 2026-02-17T06:04:35.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:35.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=223338299403 2026-02-17T06:04:35.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 223338299403' 2026-02-17T06:04:35.356 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 223338299403 2026-02-17T06:04:35.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:35.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 223338299403 -lt 223338299403 2026-02-17T06:04:35.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:35.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743426 2026-02-17T06:04:35.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:35.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:04:35.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743426 2026-02-17T06:04:35.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:35.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743426 2026-02-17T06:04:35.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743426' 2026-02-17T06:04:35.520 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 296352743426 2026-02-17T06:04:35.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:04:35.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743426 -lt 296352743426 2026-02-17T06:04:35.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:04:35.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:35.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:35.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:04:35.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:04:35.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:04:35.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:04:35.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:04:35.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:04:35.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:04:35.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:04:36.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:04:36.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:04:36.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:36.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:36.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:04:36.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:04:36.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:04:36.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3647: corrupt_scrub_erasure: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ5 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ5 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ5 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:04:36.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:04:36.397 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:04:36.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:04:36.601 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:04:36.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:04:36.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:04:36.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:04:36.634 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 5 % 2 2026-02-17T06:04:36.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:04:36.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:04:36.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3652: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=2 2026-02-17T06:04:36.638 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records in 2026-02-17T06:04:36.638 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records out 2026-02-17T06:04:36.638 INFO:tasks.workunit.client.0.trial074.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000100028 s, 40.9 MB/s 2026-02-17T06:04:36.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3653: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:04:36.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:04:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:04:36.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:04:36.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:04:37.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:04:37.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:04:37.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:04:37.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:04:37.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:04:37.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:04:37.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:04:37.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:04:39.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:04:39.123 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:04:39.124 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:04:39.124 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:04:39.124 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:04:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:04:39.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:04:39.126 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:04:39.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:04:39.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:04:39.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:04:39.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:04:39.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:04:39.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:04:39.149 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:39.138+0000 7fabba73c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:39.151 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:39.138+0000 7fabba73c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:39.152 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:39.138+0000 7fabba73c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:39.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:04:39.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:04:39.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:04:39.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:04:39.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:04:39.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:39.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:04:39.279 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:04:39.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:39.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:39.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:40.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:40.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:40.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:04:40.431 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:40.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:40.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:40.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:40.622 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:40.610+0000 7fabba73c8c0 -1 Falling back to public interface 2026-02-17T06:04:41.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:41.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:41.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:04:41.586 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:04:41.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:41.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:41.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:42.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:42.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:42.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:04:42.741 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:04:42.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:42.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:42.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:43.638 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:43.626+0000 7fabba73c8c0 -1 osd.1 72 log_to_monitors true 2026-02-17T06:04:43.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:43.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:43.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:04:43.898 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:04:43.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:43.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:04:44.044 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 76 up_thru 76 down_at 73 last_clean_interval [52,72) [v2:127.0.0.1:6810/618481170,v1:127.0.0.1:6811/618481170] [v2:127.0.0.1:6812/618481170,v1:127.0.0.1:6813/618481170] exists,up 720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:04:44.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:04:44.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:04:44.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:04:44.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:04:44.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:04:44.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:04:44.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:04:44.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:04:44.045 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:04:44.045 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:04:44.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:04:44.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:04:44.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:04:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:04:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:04:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:04:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:04:44.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:04:44.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:04:44.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:04:44.304 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:44.304 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:04:44.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:04:44.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:44.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:04:44.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=253403070475 2026-02-17T06:04:44.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 253403070475 2026-02-17T06:04:44.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070475' 2026-02-17T06:04:44.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:44.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:04:44.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=326417514498 2026-02-17T06:04:44.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 326417514498 2026-02-17T06:04:44.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070475 1-326417514498' 2026-02-17T06:04:44.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:44.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:04:44.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743429 2026-02-17T06:04:44.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743429 2026-02-17T06:04:44.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-253403070475 1-326417514498 2-296352743429' 2026-02-17T06:04:44.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:44.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-253403070475 2026-02-17T06:04:44.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:44.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:04:44.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-253403070475 2026-02-17T06:04:44.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:44.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=253403070475 2026-02-17T06:04:44.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 253403070475' 2026-02-17T06:04:44.522 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 253403070475 2026-02-17T06:04:44.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:44.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070474 -lt 253403070475 2026-02-17T06:04:44.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:45.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:04:45.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:45.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070474 -lt 253403070475 2026-02-17T06:04:45.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:46.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:04:46.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:46.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 253403070475 -lt 253403070475 2026-02-17T06:04:46.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:46.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-326417514498 2026-02-17T06:04:46.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:46.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:04:46.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-326417514498 2026-02-17T06:04:46.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:46.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=326417514498 2026-02-17T06:04:46.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 326417514498' 2026-02-17T06:04:46.990 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 326417514498 2026-02-17T06:04:46.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:47.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514498 -lt 326417514498 2026-02-17T06:04:47.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:47.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743429 2026-02-17T06:04:47.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:47.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:04:47.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743429 2026-02-17T06:04:47.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:47.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743429 2026-02-17T06:04:47.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743429' 2026-02-17T06:04:47.157 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 296352743429 2026-02-17T06:04:47.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:04:47.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743429 -lt 296352743429 2026-02-17T06:04:47.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:04:47.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:47.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:47.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:04:47.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:04:47.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:04:47.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:04:47.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:04:47.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:04:47.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:04:47.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:04:47.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:04:47.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:04:47.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:47.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:47.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:04:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:04:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:04:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:04:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ6 2026-02-17T06:04:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ6 2026-02-17T06:04:47.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:04:47.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:04:47.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ6 2026-02-17T06:04:47.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:04:47.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:04:47.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:04:48.060 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:04:48.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:04:48.265 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:04:48.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:04:48.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:04:48.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:04:48.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 6 % 2 2026-02-17T06:04:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T06:04:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:04:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3657: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:04:48.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:04:48.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:04:48.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:04:48.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:04:48.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:04:49.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:04:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:04:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:04:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:04:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:04:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:04:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:04:49.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ6 rm-attr hinfo_key 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:04:50.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:04:50.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:04:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:04:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:04:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:04:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:04:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:04:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:04:50.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:04:50.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:04:50.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:04:50.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:04:50.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:04:50.830 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:04:50.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:04:50.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:04:50.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:04:50.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:04:50.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:04:50.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:04:50.853 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:50.843+0000 7faa632518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:50.854 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:50.843+0000 7faa632518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:50.855 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:50.843+0000 7faa632518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:04:50.978 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:04:50.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:50.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:51.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:52.066 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:52.055+0000 7faa632518c0 -1 Falling back to public interface 2026-02-17T06:04:52.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:52.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:52.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:04:52.133 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:52.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:52.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:52.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:53.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:53.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:53.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:04:53.283 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:04:53.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:53.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:53.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:54.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:54.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:54.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:04:54.438 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:04:54.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:54.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:54.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:04:55.047 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:04:55.035+0000 7faa632518c0 -1 osd.0 79 log_to_monitors true 2026-02-17T06:04:55.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:04:55.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:04:55.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:04:55.592 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:04:55.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:04:55.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:04:55.743 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [59,79) [v2:127.0.0.1:6802/860903465,v1:127.0.0.1:6803/860903465] [v2:127.0.0.1:6804/860903465,v1:127.0.0.1:6805/860903465] exists,up 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:04:55.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:04:55.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:04:55.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:04:55.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:04:55.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:04:55.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:04:55.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:04:55.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:04:55.744 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:04:55.744 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:04:55.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:04:55.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:04:55.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:04:55.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:04:55.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:04:55.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:04:55.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:04:55.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:04:55.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:04:56.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:04:56.009 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:04:56.009 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:04:56.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:04:56.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:56.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:04:56.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285570 2026-02-17T06:04:56.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285570 2026-02-17T06:04:56.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-356482285570' 2026-02-17T06:04:56.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:56.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:04:56.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=326417514501 2026-02-17T06:04:56.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 326417514501 2026-02-17T06:04:56.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-356482285570 1-326417514501' 2026-02-17T06:04:56.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:04:56.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:04:56.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743432 2026-02-17T06:04:56.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743432 2026-02-17T06:04:56.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-356482285570 1-326417514501 2-296352743432' 2026-02-17T06:04:56.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:56.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-356482285570 2026-02-17T06:04:56.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:56.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:04:56.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-356482285570 2026-02-17T06:04:56.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:56.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285570 2026-02-17T06:04:56.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 356482285570' 2026-02-17T06:04:56.228 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 356482285570 2026-02-17T06:04:56.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:04:56.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285570 -lt 356482285570 2026-02-17T06:04:56.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:56.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-326417514501 2026-02-17T06:04:56.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:56.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:04:56.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-326417514501 2026-02-17T06:04:56.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:56.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=326417514501 2026-02-17T06:04:56.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 326417514501' 2026-02-17T06:04:56.396 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 326417514501 2026-02-17T06:04:56.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:56.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514500 -lt 326417514501 2026-02-17T06:04:56.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:57.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:04:57.552 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:57.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514500 -lt 326417514501 2026-02-17T06:04:57.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:04:58.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:04:58.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:04:58.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514501 -lt 326417514501 2026-02-17T06:04:58.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:04:58.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743432 2026-02-17T06:04:58.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:04:58.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:04:58.862 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743432 2026-02-17T06:04:58.862 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:04:58.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743432 2026-02-17T06:04:58.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743432' 2026-02-17T06:04:58.864 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 296352743432 2026-02-17T06:04:58.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:04:59.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743433 -lt 296352743432 2026-02-17T06:04:59.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:04:59.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:59.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:59.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:04:59.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:04:59.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:04:59.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:04:59.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:04:59.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:04:59.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:04:59.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:04:59.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:04:59.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:04:59.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:04:59.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:04:59.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3658: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3659: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:04:59.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:04:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:05:00.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:05:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:02.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:05:02.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:05:02.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:05:02.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:05:02.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:05:02.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:05:02.041 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:05:02.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:05:02.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:05:02.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:05:02.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:05:02.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:05:02.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:05:02.060 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:02.047+0000 7ff3eb8148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:02.061 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:02.051+0000 7ff3eb8148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:02.062 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:02.051+0000 7ff3eb8148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:02.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:05:02.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:05:02.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:05:02.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:05:02.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:05:02.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:02.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:05:02.197 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:05:02.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:02.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:02.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:03.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:03.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:03.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:05:03.351 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:03.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:03.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:03.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:03.545 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:03.535+0000 7ff3eb8148c0 -1 Falling back to public interface 2026-02-17T06:05:04.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:04.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:04.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:05:04.504 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:05:04.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:04.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:04.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:05.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:05.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:05.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:05:05.656 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:05:05.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:05.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:05.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:06.508 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:06.495+0000 7ff3eb8148c0 -1 osd.1 84 log_to_monitors true 2026-02-17T06:05:06.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:06.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:06.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:05:06.807 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:05:06.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:06.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:06.959 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 88 up_thru 88 down_at 85 last_clean_interval [76,84) [v2:127.0.0.1:6810/3052101064,v1:127.0.0.1:6811/3052101064] [v2:127.0.0.1:6812/3052101064,v1:127.0.0.1:6813/3052101064] exists,up 720800fd-904c-4c90-946c-3f8dcccbe5f5 2026-02-17T06:05:06.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:05:06.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:05:06.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:05:06.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:05:06.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:05:06.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:05:06.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:05:06.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:05:06.960 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:05:06.960 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:05:06.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:05:06.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:05:06.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:05:07.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:05:07.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:05:07.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:05:07.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:05:07.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:05:07.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:05:07.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:05:07.222 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:07.222 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:05:07.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:05:07.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:07.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:05:07.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285573 2026-02-17T06:05:07.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285573 2026-02-17T06:05:07.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-356482285573' 2026-02-17T06:05:07.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:07.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:05:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122050 2026-02-17T06:05:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122050 2026-02-17T06:05:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-356482285573 1-377957122050' 2026-02-17T06:05:07.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:07.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:05:07.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743435 2026-02-17T06:05:07.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743435 2026-02-17T06:05:07.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-356482285573 1-377957122050 2-296352743435' 2026-02-17T06:05:07.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:07.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-356482285573 2026-02-17T06:05:07.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:07.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:05:07.441 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-356482285573 2026-02-17T06:05:07.441 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:07.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285573 2026-02-17T06:05:07.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 356482285573' 2026-02-17T06:05:07.443 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 356482285573 2026-02-17T06:05:07.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:07.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285572 -lt 356482285573 2026-02-17T06:05:07.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:05:08.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:05:08.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:08.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285573 -lt 356482285573 2026-02-17T06:05:08.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:08.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122050 2026-02-17T06:05:08.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:08.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:05:08.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122050 2026-02-17T06:05:08.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:08.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122050 2026-02-17T06:05:08.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122050' 2026-02-17T06:05:08.754 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 377957122050 2026-02-17T06:05:08.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:05:08.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122050 -lt 377957122050 2026-02-17T06:05:08.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:08.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743435 2026-02-17T06:05:08.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:08.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:05:08.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743435 2026-02-17T06:05:08.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:08.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743435 2026-02-17T06:05:08.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743435' 2026-02-17T06:05:08.917 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 296352743435 2026-02-17T06:05:08.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:05:09.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743436 -lt 296352743435 2026-02-17T06:05:09.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:05:09.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:05:09.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:05:09.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:05:09.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:05:09.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:05:09.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:05:09.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:05:09.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:05:09.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:05:09.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:05:09.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:05:09.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:05:09.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:05:09.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:05:09.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:05:09.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:05:09.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:05:09.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ7 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ7 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ7 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:05:09.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:05:09.796 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:05:09.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:05:09.999 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:05:10.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:05:10.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:05:10.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:05:10.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 7 % 2 2026-02-17T06:05:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T06:05:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T06:05:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3663: corrupt_scrub_erasure: local payload=MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T06:05:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3664: corrupt_scrub_erasure: echo MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T06:05:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3665: corrupt_scrub_erasure: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/DIFFERENT 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3668: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:05:10.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:05:10.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:05:10.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:05:10.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:05:10.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:05:10.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:05:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ1 get-attr hinfo_key 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:05:12.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:05:12.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:05:12.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:05:12.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:05:12.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:05:12.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:05:12.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:05:12.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:05:12.232 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:05:12.233 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:05:12.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:05:12.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:05:12.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:05:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:05:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:05:12.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:05:12.251 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:12.239+0000 7f0008ced8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:12.252 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:12.243+0000 7f0008ced8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:12.253 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:12.243+0000 7f0008ced8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:05:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:12.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:12.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:13.538 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:13.527+0000 7f0008ced8c0 -1 Falling back to public interface 2026-02-17T06:05:13.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:13.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:13.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:05:13.541 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:13.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:13.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:13.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:14.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:14.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:14.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:05:14.700 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:05:14.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:14.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:14.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:15.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:15.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:15.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:05:15.851 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:05:15.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:15.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:15.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:16.496 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:16.487+0000 7f0008ced8c0 -1 osd.0 91 log_to_monitors true 2026-02-17T06:05:16.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:16.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:16.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:05:16.999 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:05:16.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:16.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:17.148 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 95 up_thru 95 down_at 92 last_clean_interval [83,91) [v2:127.0.0.1:6802/3741388652,v1:127.0.0.1:6803/3741388652] [v2:127.0.0.1:6804/3741388652,v1:127.0.0.1:6805/3741388652] exists,up 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:05:17.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:05:17.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:05:17.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:05:17.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:05:17.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:05:17.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:05:17.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:05:17.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:05:17.149 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:05:17.149 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:05:17.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:05:17.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:05:17.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:05:17.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:05:17.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:05:17.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:05:17.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:05:17.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:05:17.264 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:05:17.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:05:17.420 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:17.420 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:05:17.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:05:17.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:17.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:05:17.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=408021893122 2026-02-17T06:05:17.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 408021893122 2026-02-17T06:05:17.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-408021893122' 2026-02-17T06:05:17.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:17.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:05:17.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122053 2026-02-17T06:05:17.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122053 2026-02-17T06:05:17.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-408021893122 1-377957122053' 2026-02-17T06:05:17.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:17.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:05:17.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743438 2026-02-17T06:05:17.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743438 2026-02-17T06:05:17.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-408021893122 1-377957122053 2-296352743438' 2026-02-17T06:05:17.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:17.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-408021893122 2026-02-17T06:05:17.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:17.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:05:17.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-408021893122 2026-02-17T06:05:17.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:17.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=408021893122 2026-02-17T06:05:17.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 408021893122' 2026-02-17T06:05:17.641 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 408021893122 2026-02-17T06:05:17.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:17.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285573 -lt 408021893122 2026-02-17T06:05:17.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:05:18.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:05:18.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:18.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893122 -lt 408021893122 2026-02-17T06:05:18.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:18.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122053 2026-02-17T06:05:18.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:18.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:05:18.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122053 2026-02-17T06:05:18.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:18.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122053 2026-02-17T06:05:18.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122053' 2026-02-17T06:05:18.953 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 377957122053 2026-02-17T06:05:18.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:05:19.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122053 -lt 377957122053 2026-02-17T06:05:19.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:19.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743438 2026-02-17T06:05:19.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:19.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:05:19.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743438 2026-02-17T06:05:19.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:19.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743438 2026-02-17T06:05:19.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743438' 2026-02-17T06:05:19.109 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 296352743438 2026-02-17T06:05:19.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:05:19.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743439 -lt 296352743438 2026-02-17T06:05:19.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:05:19.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:05:19.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:05:19.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:05:19.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:05:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:05:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:05:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:05:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:05:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:05:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:05:19.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:05:19.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:05:19.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:05:19.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:05:19.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:05:19.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3669: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:05:19.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:05:21.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:05:22.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:05:22.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:05:22.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:05:22.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:05:22.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:05:22.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:05:22.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:05:22.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:05:22.293 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:22.283+0000 7f633af538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:22.294 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:22.283+0000 7f633af538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:22.295 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:22.283+0000 7f633af538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:05:22.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:22.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:22.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:23.574 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:23.563+0000 7f633af538c0 -1 Falling back to public interface 2026-02-17T06:05:23.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:23.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:23.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:05:23.575 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:23.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:23.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:23.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:24.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:24.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:24.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:05:24.726 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:05:24.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:24.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:24.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:25.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:25.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:25.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:05:25.881 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:05:25.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:25.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:26.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:26.553 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:26.543+0000 7f633af538c0 -1 osd.0 97 log_to_monitors true 2026-02-17T06:05:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:05:27.036 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:05:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:27.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:27.189 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 101 up_thru 101 down_at 98 last_clean_interval [95,97) [v2:127.0.0.1:6802/3684430576,v1:127.0.0.1:6803/3684430576] [v2:127.0.0.1:6804/3684430576,v1:127.0.0.1:6805/3684430576] exists,up 1bf6acc4-3753-42db-b389-e73235328388 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:05:27.190 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:05:27.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:05:27.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:05:27.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:05:27.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:05:27.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:05:27.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:05:27.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:05:27.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:05:27.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:05:27.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:05:27.467 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:27.467 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:05:27.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:05:27.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:27.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:05:27.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696898 2026-02-17T06:05:27.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696898 2026-02-17T06:05:27.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696898' 2026-02-17T06:05:27.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:27.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:05:27.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122056 2026-02-17T06:05:27.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122056 2026-02-17T06:05:27.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696898 1-377957122056' 2026-02-17T06:05:27.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:27.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:05:27.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743441 2026-02-17T06:05:27.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743441 2026-02-17T06:05:27.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696898 1-377957122056 2-296352743441' 2026-02-17T06:05:27.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:27.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-433791696898 2026-02-17T06:05:27.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:27.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:05:27.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-433791696898 2026-02-17T06:05:27.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:27.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696898 2026-02-17T06:05:27.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 433791696898' 2026-02-17T06:05:27.685 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 433791696898 2026-02-17T06:05:27.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:27.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 408021893122 -lt 433791696898 2026-02-17T06:05:27.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:05:28.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:05:28.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:28.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696898 -lt 433791696898 2026-02-17T06:05:28.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:28.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122056 2026-02-17T06:05:28.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:05:28.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122056 2026-02-17T06:05:28.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:28.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122056 2026-02-17T06:05:28.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122056' 2026-02-17T06:05:28.999 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 377957122056 2026-02-17T06:05:28.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:05:29.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122056 -lt 377957122056 2026-02-17T06:05:29.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:29.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743441 2026-02-17T06:05:29.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:29.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:05:29.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743441 2026-02-17T06:05:29.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:29.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743441 2026-02-17T06:05:29.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743441' 2026-02-17T06:05:29.168 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 296352743441 2026-02-17T06:05:29.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:05:29.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743442 -lt 296352743441 2026-02-17T06:05:29.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:05:29.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:05:29.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:05:29.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:05:29.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:05:29.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:05:29.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:05:29.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:05:29.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:05:29.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:05:29.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:05:29.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:05:29.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:05:29.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:05:29.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:05:29.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:05:29.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:05:29.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:05:29.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3670: corrupt_scrub_erasure: rm -f td/osd-scrub-repair/hinfo 2026-02-17T06:05:29.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: get_pg ecpool EOBJ0 2026-02-17T06:05:29.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T06:05:29.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=EOBJ0 2026-02-17T06:05:29.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool EOBJ0 2026-02-17T06:05:29.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:05:29.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: local pg=3.0 2026-02-17T06:05:29.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3678: corrupt_scrub_erasure: pg_scrub 3.0 2026-02-17T06:05:29.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T06:05:29.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T06:05:29.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:05:29.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:05:30.000 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:05:30.000 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:05:30.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:05:30.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:05:30.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:05:30.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:05:30.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:05:30.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:05:30.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:05:30.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:05:30.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:05:30.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:05:30.448 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:30.448 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:05:30.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:05:30.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:30.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:05:30.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696899 2026-02-17T06:05:30.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696899 2026-02-17T06:05:30.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696899' 2026-02-17T06:05:30.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:30.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:05:30.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122057 2026-02-17T06:05:30.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122057 2026-02-17T06:05:30.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696899 1-377957122057' 2026-02-17T06:05:30.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:30.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:05:30.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743443 2026-02-17T06:05:30.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743443 2026-02-17T06:05:30.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696899 1-377957122057 2-296352743443' 2026-02-17T06:05:30.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:30.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-433791696899 2026-02-17T06:05:30.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:30.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:05:30.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-433791696899 2026-02-17T06:05:30.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:30.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696899 2026-02-17T06:05:30.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 433791696899' 2026-02-17T06:05:30.669 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 433791696899 2026-02-17T06:05:30.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:30.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696898 -lt 433791696899 2026-02-17T06:05:30.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:05:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:05:31.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:31.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696898 -lt 433791696899 2026-02-17T06:05:31.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:05:32.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:05:32.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:33.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696900 -lt 433791696899 2026-02-17T06:05:33.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:33.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122057 2026-02-17T06:05:33.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:05:33.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122057 2026-02-17T06:05:33.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:33.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122057 2026-02-17T06:05:33.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122057' 2026-02-17T06:05:33.135 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 377957122057 2026-02-17T06:05:33.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:05:33.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122058 -lt 377957122057 2026-02-17T06:05:33.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:33.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743443 2026-02-17T06:05:33.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:33.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:05:33.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743443 2026-02-17T06:05:33.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:33.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743443 2026-02-17T06:05:33.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743443' 2026-02-17T06:05:33.298 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 296352743443 2026-02-17T06:05:33.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:05:33.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743443 -lt 296352743443 2026-02-17T06:05:33.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:05:33.458 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:05:33.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:05:33.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:05:33.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:05:33.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:05:33.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:05:33.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:05:33.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:05:33.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:05:33.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:05:33.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:05:33.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T06:05:33.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:33.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:05:33.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:33.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:05:33.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:33.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T06:05:33.823 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0s0 on osd.1 to scrub 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:33.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:05:33.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:33.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:05:33.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:03:27.671060+0000 '>' 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:33.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:05:34.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:05:34.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:05:34.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:05:34.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:34.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:05:34.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:34.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:05:35.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:03:27.671060+0000 '>' 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:35.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:05:36.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:05:36.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:05:36.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:05:36.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:36.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:05:36.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:36.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:05:36.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:03:27.671060+0000 '>' 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:36.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:05:37.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:05:37.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:05:37.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:05:37.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:37.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:05:37.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:37.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:05:37.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:03:27.671060+0000 '>' 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:37.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:05:38.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:05:38.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:05:38.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:05:38.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:38.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:05:38.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:05:38.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:38.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:05:34.464159+0000 '>' 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:38.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:05:38.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3680: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T06:05:38.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T06:05:38.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: test 1 = 1 2026-02-17T06:05:38.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T06:05:38.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T06:05:38.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3686: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T06:05:38.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T06:05:38.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: epoch=101 2026-02-17T06:05:38.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:05:38.680 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:05:38.680 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:05:38.680 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:05:38.680 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:05:38.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:05:38.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:05:38.696 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:05:38.696 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:05:38.696 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:05:38.696 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:05:38.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:05:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4311: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:05:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:05:38.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4312: corrupt_scrub_erasure: test no = yes 2026-02-17T06:05:38.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4317: corrupt_scrub_erasure: test '' = yes 2026-02-17T06:05:38.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4322: corrupt_scrub_erasure: pg_deep_scrub 3.0 2026-02-17T06:05:38.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T06:05:38.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T06:05:38.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:05:38.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:05:38.724 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:05:38.724 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:05:38.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:05:38.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:05:38.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:05:39.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:05:39.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:05:39.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:05:39.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:05:39.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:05:39.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:05:39.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:05:39.163 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:05:39.163 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:05:39.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:05:39.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:39.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:05:39.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696902 2026-02-17T06:05:39.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696902 2026-02-17T06:05:39.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696902' 2026-02-17T06:05:39.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:39.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:05:39.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122060 2026-02-17T06:05:39.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122060 2026-02-17T06:05:39.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696902 1-377957122060' 2026-02-17T06:05:39.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:05:39.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:05:39.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743446 2026-02-17T06:05:39.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743446 2026-02-17T06:05:39.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-433791696902 1-377957122060 2-296352743446' 2026-02-17T06:05:39.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:39.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-433791696902 2026-02-17T06:05:39.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:39.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:05:39.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-433791696902 2026-02-17T06:05:39.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:39.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696902 2026-02-17T06:05:39.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 433791696902' 2026-02-17T06:05:39.381 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 433791696902 2026-02-17T06:05:39.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:39.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696901 -lt 433791696902 2026-02-17T06:05:39.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:05:40.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:05:40.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:05:40.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696902 -lt 433791696902 2026-02-17T06:05:40.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:40.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122060 2026-02-17T06:05:40.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:40.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:05:40.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122060 2026-02-17T06:05:40.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:40.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122060 2026-02-17T06:05:40.699 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 377957122060 2026-02-17T06:05:40.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122060' 2026-02-17T06:05:40.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:05:40.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122060 -lt 377957122060 2026-02-17T06:05:40.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:05:40.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-296352743446 2026-02-17T06:05:40.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:05:40.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:05:40.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-296352743446 2026-02-17T06:05:40.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:05:40.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743446 2026-02-17T06:05:40.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 296352743446' 2026-02-17T06:05:40.855 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 296352743446 2026-02-17T06:05:40.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:05:41.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743446 -lt 296352743446 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:05:41.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:41.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:05:41.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:41.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T06:05:41.382 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0s0 on osd.1 to deep-scrub 2026-02-17T06:05:41.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T06:03:27.671060+0000 last_deep_scrub_stamp 2026-02-17T06:05:41.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:05:41.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:41.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T06:05:41.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:05:41.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:05:41.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:05:41.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:41.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:05:41.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:41.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:05:41.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:03:27.671060+0000 '>' 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:41.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:05:42.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:05:42.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:05:42.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:05:42.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:05:42.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:05:42.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:05:42.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:05:42.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:05:41.393972+0000 '>' 2026-02-17T06:03:27.671060+0000 2026-02-17T06:05:42.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:05:42.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4324: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T06:05:42.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T06:05:42.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: test 1 = 1 2026-02-17T06:05:42.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T06:05:42.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T06:05:42.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4330: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T06:05:42.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T06:05:42.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: epoch=101 2026-02-17T06:05:42.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4334: corrupt_scrub_erasure: '[' true = true ']' 2026-02-17T06:05:42.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4336: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:05:42.762 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:05:42.762 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:05:42.762 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:05:42.762 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T06:05:42.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4336: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:05:42.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4336: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:05:42.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T06:05:42.781 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:05:42.781 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:05:42.781 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:05:42.781 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:05:42.781 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:05:42.781 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:05:42.781 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:05:42.782 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:05:42.782 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:05:42.782 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:05:42.782 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:05:42.782 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T06:05:42.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T06:05:42.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:05:42.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5725: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:05:42.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:05:42.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5726: corrupt_scrub_erasure: test no = yes 2026-02-17T06:05:42.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5737: corrupt_scrub_erasure: test '' = yes 2026-02-17T06:05:42.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5742: corrupt_scrub_erasure: ceph osd pool rm ecpool ecpool --yes-i-really-really-mean-it 2026-02-17T06:05:42.991 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' removed 2026-02-17T06:05:43.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:05:43.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:05:43.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:05:43.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:05:43.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:05:43.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:05:43.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:05:43.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:05:43.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:05:43.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:05:43.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:05:43.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:05:43.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:05:43.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:05:43.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:05:43.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:05:43.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:05:43.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:05:43.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:05:43.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:05:43.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:05:43.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:05:43.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:05:43.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:05:43.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:05:43.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:05:43.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:05:43.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:05:43.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:05:43.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:05:43.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:05:43.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:05:43.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:05:43.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:05:43.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:05:43.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:05:43.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:05:43.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:05:43.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:05:43.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:05:43.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:05:43.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:05:43.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:05:43.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:05:43.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:05:43.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:05:43.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:05:43.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:05:43.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:05:43.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:05:43.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:05:43.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:05:43.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:05:43.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_replicated td/osd-scrub-repair 2026-02-17T06:05:43.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: TEST_corrupt_scrub_replicated: local dir=td/osd-scrub-repair 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:992: TEST_corrupt_scrub_replicated: local poolname=csr_pool 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:993: TEST_corrupt_scrub_replicated: local total_objs=19 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: TEST_corrupt_scrub_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:05:43.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T06:05:43.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:05:43.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:05:43.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:05:43.191 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:05:43.191 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.191 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:43.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T06:05:43.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:05:43.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:05:43.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:05:43.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:05:43.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:05:43.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:05:43.220 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:05:43.220 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:05:43.220 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:05:43.220 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:05:43.220 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.220 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.221 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:05:43.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:05:43.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:05:43.284 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:05:43.285 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:05:43.285 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.285 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.285 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:05:43.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:05:43.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:05:43.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:996: TEST_corrupt_scrub_replicated: run_mgr td/osd-scrub-repair x 2026-02-17T06:05:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:05:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:05:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:05:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:05:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:05:43.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:05:43.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:05:43.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:05:43.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:05:43.437 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:05:43.437 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.438 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:43.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:05:43.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:05:43.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:997: TEST_corrupt_scrub_replicated: run_osd td/osd-scrub-repair 0 2026-02-17T06:05:43.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:05:43.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:05:43.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:05:43.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:05:43.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:05:43.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:05:43.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:05:43.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:05:43.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:05:43.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:05:43.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:05:43.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:05:43.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:05:43.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:05:43.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:05:43.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:05:43.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:05:43.466 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:05:43.466 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:43.466 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:43.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:43.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:43.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:05:43.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:05:43.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:05:43.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:05:43.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:05:43.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:05:43.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:05:43.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:05:43.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:05:43.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:05:43.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:05:43.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:05:43.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:05:43.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:05:43.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 18ed35de-096b-4dff-b9f5-ec17fd7a8c19' 2026-02-17T06:05:43.470 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:05:43.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:05:43.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA3BZRps6+5GxAA/44NtT6aTbk8yFfLHUXYeA== 2026-02-17T06:05:43.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA3BZRps6+5GxAA/44NtT6aTbk8yFfLHUXYeA=="}' 2026-02-17T06:05:43.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:05:43.561 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:05:43.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:05:43.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA3BZRps6+5GxAA/44NtT6aTbk8yFfLHUXYeA== --osd-uuid 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:05:43.586 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:43.575+0000 7f3be3e138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:43.587 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:43.575+0000 7f3be3e138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:43.588 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:43.579+0000 7f3be3e138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:43.589 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:43.579+0000 7f3be3e138c0 -1 bdev(0x56391a931c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:05:43.589 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:43.579+0000 7f3be3e138c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:05:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:05:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:05:46.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:05:46.198 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:05:46.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:05:46.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:05:46.391 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:05:46.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:05:46.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:05:46.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:05:46.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:05:46.405 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:46.395+0000 7f9975f848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:46.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:46.395+0000 7f9975f848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:46.408 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:46.399+0000 7f9975f848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:46.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:05:46.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:05:46.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:05:46.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:05:46.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:05:46.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:46.548 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:05:46.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:05:46.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:46.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:46.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:47.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:47.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:47.700 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:05:47.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:05:47.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:47.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:47.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:48.114 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:48.103+0000 7f9975f848c0 -1 Falling back to public interface 2026-02-17T06:05:48.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:48.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:48.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:05:48.853 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:05:48.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:48.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:49.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:50.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:50.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:50.007 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:05:50.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:05:50.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:50.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:50.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:51.066 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:51.055+0000 7f9975f848c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:05:51.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:51.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:51.160 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:05:51.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:05:51.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:51.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:51.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:52.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:52.311 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:05:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:05:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:52.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:05:52.463 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1619411292,v1:127.0.0.1:6803/1619411292] [v2:127.0.0.1:6804/1619411292,v1:127.0.0.1:6805/1619411292] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: TEST_corrupt_scrub_replicated: run_osd td/osd-scrub-repair 1 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:05:52.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:05:52.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:05:52.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:05:52.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:05:52.465 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:05:52.465 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:05:52.465 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:05:52.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:05:52.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:05:52.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:05:52.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:05:52.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 85a1dc65-55a4-475c-a132-09f1a319858f' 2026-02-17T06:05:52.469 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:05:52.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:05:52.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBABZRp65pAHBAAb5uFIvPUW/sNZkgQK9hZWg== 2026-02-17T06:05:52.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBABZRp65pAHBAAb5uFIvPUW/sNZkgQK9hZWg=="}' 2026-02-17T06:05:52.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 85a1dc65-55a4-475c-a132-09f1a319858f -i td/osd-scrub-repair/1/new.json 2026-02-17T06:05:52.637 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:05:52.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:05:52.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBABZRp65pAHBAAb5uFIvPUW/sNZkgQK9hZWg== --osd-uuid 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:05:52.663 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:52.651+0000 7f9695b7c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:52.664 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:52.655+0000 7f9695b7c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:52.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:52.655+0000 7f9695b7c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:52.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:52.655+0000 7f9695b7c8c0 -1 bdev(0x55908b76dc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:05:52.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:52.655+0000 7f9695b7c8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:05:54.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:05:54.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:05:54.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:05:54.962 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:05:54.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:05:55.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:05:55.160 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:05:55.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:05:55.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:05:55.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:05:55.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:05:55.176 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:55.167+0000 7f5b67ea08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:55.177 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:55.167+0000 7f5b67ea08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:55.178 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:55.167+0000 7f5b67ea08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:05:55.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:05:55.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:05:55.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:05:55.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:05:55.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:05:55.310 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:05:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:55.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:05:55.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:55.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:55.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:56.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:56.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:56.463 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:05:56.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:05:56.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:56.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:56.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:56.689 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:56.679+0000 7f5b67ea08c0 -1 Falling back to public interface 2026-02-17T06:05:57.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:57.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:57.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:05:57.615 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:05:57.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:57.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:57.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:58.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:58.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:58.768 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:05:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:05:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:05:58.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:05:59.696 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:05:59.687+0000 7f5b67ea08c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:05:59.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:05:59.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:05:59.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:05:59.920 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:05:59.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:05:59.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:00.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:01.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:01.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:01.071 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:06:01.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:06:01.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:01.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:01.221 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1454904363,v1:127.0.0.1:6811/1454904363] [v2:127.0.0.1:6812/1454904363,v1:127.0.0.1:6813/1454904363] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:06:01.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:06:01.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:06:01.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:06:01.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: TEST_corrupt_scrub_replicated: create_rbd_pool 2026-02-17T06:06:01.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:06:01.363 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:06:01.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:06:01.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:06:01.586 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:06:01.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:06:02.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:06:02.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1000: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-17T06:06:02.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:06:02.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:06:02.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:06:02.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:06:02.879 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:06:02.879 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:06:02.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:06:02.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:06:02.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:06:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:06:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:06:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:06:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:06:02.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:06:02.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:06:03.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:06:03.150 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:06:03.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:06:03.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:03.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:06:03.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:06:03.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:06:03.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:06:03.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:03.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:06:03.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T06:06:03.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T06:06:03.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T06:06:03.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:03.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:06:03.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:03.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:06:03.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:06:03.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:03.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:06:03.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:06:03.297 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:06:03.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:03.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T06:06:03.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:06:04.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:06:04.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:04.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:06:04.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:04.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T06:06:04.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:04.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:06:04.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T06:06:04.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:04.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T06:06:04.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T06:06:04.607 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672962 2026-02-17T06:06:04.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:06:04.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T06:06:04.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:06:04.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:04.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:04.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:06:04.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:06:04.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:06:04.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:06:04.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:06:04.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:06:04.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:06:04.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:06:05.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:06:05.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:06:05.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:05.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:05.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:06:05.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:06:05.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:06:05.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1002: TEST_corrupt_scrub_replicated: create_pool foo 1 2026-02-17T06:06:05.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T06:06:05.473 INFO:tasks.workunit.client.0.trial074.stderr:pool 'foo' created 2026-02-17T06:06:05.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:06:06.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1003: TEST_corrupt_scrub_replicated: create_pool csr_pool 1 1 2026-02-17T06:06:06.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create csr_pool 1 1 2026-02-17T06:06:06.683 INFO:tasks.workunit.client.0.trial074.stderr:pool 'csr_pool' created 2026-02-17T06:06:06.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:06:07.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1004: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-17T06:06:07.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:06:07.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:06:07.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:06:07.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:06:07.696 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:06:07.696 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:06:07.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:06:07.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:06:07.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:06:07.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:06:07.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:06:07.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:06:07.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:06:07.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:06:07.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:06:07.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:06:07.970 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:06:07.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:06:07.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:07.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:06:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T06:06:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T06:06:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T06:06:08.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:08.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:06:08.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T06:06:08.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T06:06:08.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964' 2026-02-17T06:06:08.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:08.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T06:06:08.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:08.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:06:08.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T06:06:08.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:08.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T06:06:08.118 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836486 2026-02-17T06:06:08.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T06:06:08.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:08.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T06:06:08.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:06:09.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:06:09.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:09.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T06:06:09.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:06:10.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:06:10.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:10.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T06:06:10.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:10.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T06:06:10.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:10.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:06:10.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T06:06:10.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:10.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T06:06:10.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T06:06:10.589 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T06:06:10.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:06:10.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T06:06:10.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:06:10.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:10.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:10.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:06:10.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:06:10.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:06:10.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:06:10.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:06:10.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:06:10.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:06:10.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:06:11.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:06:11.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:06:11.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:11.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:11.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:06:11.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:06:11.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:06:11.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: seq 1 19 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ1 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ1 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ1 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:11.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:11.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:11.465 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:11.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:11.669 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:11.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:11.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:11.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:11.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ1 hdr-ROBJ1 2026-02-17T06:06:11.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ1 key-ROBJ1 val-ROBJ1 2026-02-17T06:06:11.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:11.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ2 2026-02-17T06:06:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ2 2026-02-17T06:06:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ2 2026-02-17T06:06:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:11.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:11.959 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:11.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:12.163 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:12.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:12.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:12.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:12.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ2 hdr-ROBJ2 2026-02-17T06:06:12.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ2 key-ROBJ2 val-ROBJ2 2026-02-17T06:06:12.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:12.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ3 2026-02-17T06:06:12.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ3 2026-02-17T06:06:12.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:12.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:12.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ3 2026-02-17T06:06:12.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:12.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:12.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:12.451 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:12.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:12.654 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:12.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:12.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:12.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:12.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ3 hdr-ROBJ3 2026-02-17T06:06:12.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ3 key-ROBJ3 val-ROBJ3 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ4 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ4 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ4 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:12.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:12.943 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:12.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:13.147 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:13.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:13.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:13.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:13.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ4 hdr-ROBJ4 2026-02-17T06:06:13.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ4 key-ROBJ4 val-ROBJ4 2026-02-17T06:06:13.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ5 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ5 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ5 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:13.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:13.429 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:13.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:13.634 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:13.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:13.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:13.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:13.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ5 hdr-ROBJ5 2026-02-17T06:06:13.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ5 key-ROBJ5 val-ROBJ5 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ6 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ6 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ6 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:13.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:13.917 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:13.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:14.122 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:14.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:14.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:14.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:14.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ6 hdr-ROBJ6 2026-02-17T06:06:14.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ6 key-ROBJ6 val-ROBJ6 2026-02-17T06:06:14.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ7 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ7 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ7 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:14.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:14.397 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:14.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:14.602 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:14.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:14.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:14.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:14.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ7 hdr-ROBJ7 2026-02-17T06:06:14.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ7 key-ROBJ7 val-ROBJ7 2026-02-17T06:06:14.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ8 2026-02-17T06:06:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ8 2026-02-17T06:06:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ8 2026-02-17T06:06:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:14.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:14.892 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:14.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:15.097 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:15.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:15.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:15.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ8 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:15.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ8 hdr-ROBJ8 2026-02-17T06:06:15.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ8 key-ROBJ8 val-ROBJ8 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ9 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ9 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:15.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:15.391 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:15.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:15.597 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:15.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:15.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:15.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:15.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ9 hdr-ROBJ9 2026-02-17T06:06:15.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ9 key-ROBJ9 val-ROBJ9 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ10 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ10 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:15.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:15.876 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:15.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:16.081 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:16.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:16.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:16.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ10 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:16.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ10 hdr-ROBJ10 2026-02-17T06:06:16.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ10 key-ROBJ10 val-ROBJ10 2026-02-17T06:06:16.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:16.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ11 2026-02-17T06:06:16.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ11 2026-02-17T06:06:16.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:16.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:16.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ11 2026-02-17T06:06:16.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:16.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:16.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:16.348 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:16.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:16.552 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:16.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:16.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:16.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ11 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:16.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ11 hdr-ROBJ11 2026-02-17T06:06:16.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ11 key-ROBJ11 val-ROBJ11 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ12 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ12 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ12 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:16.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:16.825 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:16.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:17.031 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:17.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:17.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:17.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ12 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:17.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ12 hdr-ROBJ12 2026-02-17T06:06:17.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ12 key-ROBJ12 val-ROBJ12 2026-02-17T06:06:17.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ13 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ13 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ13 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:17.318 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:17.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:17.524 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:17.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:17.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:17.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ13 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:17.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ13 hdr-ROBJ13 2026-02-17T06:06:17.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ13 key-ROBJ13 val-ROBJ13 2026-02-17T06:06:17.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:17.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ14 2026-02-17T06:06:17.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ14 2026-02-17T06:06:17.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:17.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:17.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ14 2026-02-17T06:06:17.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:17.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:17.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:17.825 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:17.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:18.029 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:18.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:18.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:18.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ14 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:18.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ14 hdr-ROBJ14 2026-02-17T06:06:18.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ14 key-ROBJ14 val-ROBJ14 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ15 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ15 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ15 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:18.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:18.332 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:18.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:18.534 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:18.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:18.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:18.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ15 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:18.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ15 hdr-ROBJ15 2026-02-17T06:06:18.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ15 key-ROBJ15 val-ROBJ15 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ16 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ16 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ16 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:18.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:18.816 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:18.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:19.021 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:19.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:19.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:19.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ16 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:19.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ16 hdr-ROBJ16 2026-02-17T06:06:19.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ16 key-ROBJ16 val-ROBJ16 2026-02-17T06:06:19.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:19.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ17 2026-02-17T06:06:19.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ17 2026-02-17T06:06:19.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:19.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:19.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ17 2026-02-17T06:06:19.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:19.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:19.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:19.305 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:19.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:19.509 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:19.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:19.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:19.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ17 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:19.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ17 hdr-ROBJ17 2026-02-17T06:06:19.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ17 key-ROBJ17 val-ROBJ17 2026-02-17T06:06:19.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:19.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ18 2026-02-17T06:06:19.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ18 2026-02-17T06:06:19.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:19.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:19.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ18 2026-02-17T06:06:19.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:19.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:19.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:19.811 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:19.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:20.015 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:20.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:20.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:20.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ18 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:20.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ18 hdr-ROBJ18 2026-02-17T06:06:20.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ18 key-ROBJ18 val-ROBJ18 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ19 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ19 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ19 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:06:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:06:20.300 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:06:20.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:06:20.504 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:06:20.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:06:20.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:06:20.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:06:20.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ19 hdr-ROBJ19 2026-02-17T06:06:20.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ19 key-ROBJ19 val-ROBJ19 2026-02-17T06:06:20.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1015: TEST_corrupt_scrub_replicated: dd if=/dev/zero of=td/osd-scrub-repair/new.ROBJ19 bs=1024 count=1025 2026-02-17T06:06:20.574 INFO:tasks.workunit.client.0.trial074.stderr:1025+0 records in 2026-02-17T06:06:20.574 INFO:tasks.workunit.client.0.trial074.stderr:1025+0 records out 2026-02-17T06:06:20.574 INFO:tasks.workunit.client.0.trial074.stderr:1049600 bytes (1.0 MB, 1.0 MiB) copied, 0.00222793 s, 471 MB/s 2026-02-17T06:06:20.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/new.ROBJ19 2026-02-17T06:06:20.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ19 2026-02-17T06:06:20.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1019: TEST_corrupt_scrub_replicated: get_pg csr_pool ROBJ0 2026-02-17T06:06:20.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=csr_pool 2026-02-17T06:06:20.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=ROBJ0 2026-02-17T06:06:20.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-17T06:06:20.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:06:20.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1019: TEST_corrupt_scrub_replicated: local pg=3.0 2026-02-17T06:06:20.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1020: TEST_corrupt_scrub_replicated: get_primary csr_pool ROBJ0 2026-02-17T06:06:20.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=csr_pool 2026-02-17T06:06:20.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=ROBJ0 2026-02-17T06:06:20.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map csr_pool ROBJ0 2026-02-17T06:06:20.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:06:20.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1020: TEST_corrupt_scrub_replicated: local primary=1 2026-02-17T06:06:20.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: TEST_corrupt_scrub_replicated: get_asok_path osd.0 2026-02-17T06:06:20.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:06:20.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:06:20.905 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:06:20.905 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:06:20.905 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:06:20.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:06:20.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: TEST_corrupt_scrub_replicated: CEPH_ARGS= 2026-02-17T06:06:20.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: TEST_corrupt_scrub_replicated: ceph daemon /tmp/ceph-asok.40489/ceph-osd.0.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-17T06:06:20.959 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:06:20.960 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-17T06:06:20.960 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:06:20.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1025: TEST_corrupt_scrub_replicated: get_asok_path osd.1 2026-02-17T06:06:20.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:06:20.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:06:20.967 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:06:20.967 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:06:20.967 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:06:20.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:06:20.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1025: TEST_corrupt_scrub_replicated: CEPH_ARGS= 2026-02-17T06:06:20.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1025: TEST_corrupt_scrub_replicated: ceph daemon /tmp/ceph-asok.40489/ceph-osd.1.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-17T06:06:21.020 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:06:21.020 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-17T06:06:21.020 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:06:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1027: TEST_corrupt_scrub_replicated: pg_deep_scrub 3.0 2026-02-17T06:06:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T06:06:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T06:06:21.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:06:21.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:06:21.028 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:06:21.028 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:06:21.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:06:21.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:06:21.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:06:21.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:06:21.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:06:21.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:06:21.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:06:21.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:06:21.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:06:21.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:06:21.455 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:06:21.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:06:21.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:21.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:06:21.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T06:06:21.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T06:06:21.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T06:06:21.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:21.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:06:21.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672968 2026-02-17T06:06:21.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672968 2026-02-17T06:06:21.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2026-02-17T06:06:21.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:21.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T06:06:21.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:21.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:06:21.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T06:06:21.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:21.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T06:06:21.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T06:06:21.601 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836489 2026-02-17T06:06:21.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:21.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T06:06:21.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:21.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672968 2026-02-17T06:06:21.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:21.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:06:21.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672968 2026-02-17T06:06:21.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:21.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672968 2026-02-17T06:06:21.767 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672968 2026-02-17T06:06:21.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-17T06:06:21.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:06:21.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-17T06:06:21.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:06:21.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:06:21.918 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:06:21.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:06:21.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:06:21.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:06:21.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:06:21.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:06:21.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:06:21.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:06:21.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:06:21.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:06:21.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:06:21.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:06:21.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:06:21.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:06:21.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:06:22.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:22.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T06:06:22.282 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-17T06:06:22.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T06:06:06.675903+0000 last_deep_scrub_stamp 2026-02-17T06:06:22.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:06:22.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:22.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T06:06:22.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:06:22.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:06:22.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:06:22.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:06:22.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:06:22.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:06:22.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:06:22.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:06.675903+0000 '>' 2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:22.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:06:23.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:06:23.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:06:23.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:06:23.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:06:23.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:06:23.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:06:23.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:06:23.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:06.675903+0000 '>' 2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:23.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:06:24.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:06:24.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:06:24.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:06:24.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:06:24.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:06:24.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:06:24.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:06:24.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:06.675903+0000 '>' 2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:24.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:06:25.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:06:25.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:06:25.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:06:25.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:06:25.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:06:25.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:06:25.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:06:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:06.675903+0000 '>' 2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:25.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:06:26.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:06:26.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:06:26.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:06:26.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:06:26.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:06:26.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:06:26.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:06:27.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:06.675903+0000 '>' 2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:27.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:06:28.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:06:28.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:06:28.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:06:28.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:06:28.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:06:28.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:06:28.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:06:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:06.675903+0000 2026-02-17T06:06:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:06:28.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: seq 1 19 2026-02-17T06:06:28.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:28.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ1 2026-02-17T06:06:28.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 1 % 2 2026-02-17T06:06:28.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:06:28.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:06:28.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1038: TEST_corrupt_scrub_replicated: local payload=UVWXYZZZ 2026-02-17T06:06:28.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1039: TEST_corrupt_scrub_replicated: echo UVWXYZZZ 2026-02-17T06:06:28.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1040: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:06:28.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:06:28.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:06:28.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:06:28.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:06:29.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:06:29.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:29.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:06:29.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:06:29.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:06:29.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:06:29.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:06:29.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:30.196 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#-5:00000000:::scrub_3.0:head#, (61) No data available 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:06:30.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:06:30.749 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:06:30.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:06:30.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:06:30.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:06:30.752 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:06:30.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:06:30.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:06:30.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:06:30.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:06:30.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:06:30.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:06:30.771 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:30.763+0000 7fce2828c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:30.772 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:30.763+0000 7fce2828c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:30.773 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:30.763+0000 7fce2828c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:30.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:31.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:32.024 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:32.015+0000 7fce2828c8c0 -1 Falling back to public interface 2026-02-17T06:06:32.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:32.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:32.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:06:32.052 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:06:32.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:32.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:32.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:33.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:33.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:33.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:06:33.201 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:06:33.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:33.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:33.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:34.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:34.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:34.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:06:34.356 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:06:34.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:34.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:34.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:34.980 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:34.971+0000 7fce2828c8c0 -1 osd.1 62 log_to_monitors true 2026-02-17T06:06:35.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:35.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:35.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:06:35.508 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:06:35.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:35.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 66 up_thru 66 down_at 63 last_clean_interval [10,62) [v2:127.0.0.1:6810/4118393237,v1:127.0.0.1:6811/4118393237] [v2:127.0.0.1:6812/4118393237,v1:127.0.0.1:6813/4118393237] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:06:35.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:06:35.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:06:35.656 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:06:35.656 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:06:35.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:06:35.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:06:35.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:06:35.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:06:35.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:06:35.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:06:35.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:06:35.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:06:35.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:06:35.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:06:35.922 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:06:35.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:06:35.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:35.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:06:35.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-17T06:06:35.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-17T06:06:35.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-17T06:06:35.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:35.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:06:36.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841538 2026-02-17T06:06:36.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841538 2026-02-17T06:06:36.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-283467841538' 2026-02-17T06:06:36.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:36.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-17T06:06:36.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:36.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:06:36.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-17T06:06:36.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:36.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-17T06:06:36.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-17T06:06:36.070 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836493 2026-02-17T06:06:36.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:36.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-17T06:06:36.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:06:37.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:06:37.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:37.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-17T06:06:37.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:06:38.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:06:38.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:38.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 21474836493 2026-02-17T06:06:38.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:38.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-283467841538 2026-02-17T06:06:38.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:38.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:06:38.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-283467841538 2026-02-17T06:06:38.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:38.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841538 2026-02-17T06:06:38.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 283467841538' 2026-02-17T06:06:38.536 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 283467841538 2026-02-17T06:06:38.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:06:38.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841538 -lt 283467841538 2026-02-17T06:06:38.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:06:38.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:38.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:38.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:06:38.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:06:38.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:06:38.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:06:38.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:06:38.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:06:38.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:06:38.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:06:39.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:06:39.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:06:39.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:39.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:39.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:06:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:06:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:06:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ2 2026-02-17T06:06:39.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 2 % 2 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1045: TEST_corrupt_scrub_replicated: local payload=UVWXYZ 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1046: TEST_corrupt_scrub_replicated: echo UVWXYZ 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1047: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:39.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:06:39.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:06:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:06:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:06:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:06:40.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:06:40.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:06:40.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:06:40.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:41.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:06:41.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:06:41.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:06:41.678 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:06:41.678 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:06:41.678 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:06:41.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:06:41.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:06:41.680 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:06:41.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:06:41.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:06:41.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:06:41.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:06:41.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:06:41.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:06:41.702 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:41.691+0000 7f4ececd48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:41.704 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:41.695+0000 7f4ececd48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:41.705 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:41.695+0000 7f4ececd48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:41.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:06:41.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:42.948 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:42.939+0000 7f4ececd48c0 -1 Falling back to public interface 2026-02-17T06:06:42.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:42.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:42.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:06:42.984 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:06:42.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:42.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:06:43.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:44.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:44.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:44.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:06:44.139 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:06:44.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:44.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:06:44.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:45.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:45.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:45.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:06:45.293 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:06:45.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:45.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:06:45.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:45.929 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:45.919+0000 7f4ececd48c0 -1 osd.0 68 log_to_monitors true 2026-02-17T06:06:46.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:46.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:46.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:06:46.449 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:06:46.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:46.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:06:46.599 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 72 up_thru 72 down_at 69 last_clean_interval [5,68) [v2:127.0.0.1:6802/1044695102,v1:127.0.0.1:6803/1044695102] [v2:127.0.0.1:6804/1044695102,v1:127.0.0.1:6805/1044695102] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:06:46.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:06:46.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:06:46.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:06:46.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:06:46.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:06:46.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:06:46.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:06:46.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:06:46.601 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:06:46.601 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:06:46.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:06:46.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:06:46.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:06:46.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:06:46.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:06:46.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:06:46.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:06:46.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:06:46.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:06:46.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:06:46.862 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:06:46.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:06:46.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:46.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:06:46.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645314 2026-02-17T06:06:46.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645314 2026-02-17T06:06:46.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645314' 2026-02-17T06:06:46.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:46.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:06:47.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841541 2026-02-17T06:06:47.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841541 2026-02-17T06:06:47.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645314 1-283467841541' 2026-02-17T06:06:47.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:47.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-309237645314 2026-02-17T06:06:47.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:47.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:06:47.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-309237645314 2026-02-17T06:06:47.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:47.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645314 2026-02-17T06:06:47.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 309237645314' 2026-02-17T06:06:47.010 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 309237645314 2026-02-17T06:06:47.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:47.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 309237645314 2026-02-17T06:06:47.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:06:48.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:06:48.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:48.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645314 -lt 309237645314 2026-02-17T06:06:48.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:48.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-283467841541 2026-02-17T06:06:48.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:48.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:06:48.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-283467841541 2026-02-17T06:06:48.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:48.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841541 2026-02-17T06:06:48.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 283467841541' 2026-02-17T06:06:48.326 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 283467841541 2026-02-17T06:06:48.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:06:48.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841541 -lt 283467841541 2026-02-17T06:06:48.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:06:48.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:48.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:48.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:06:48.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:06:48.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:06:48.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:06:48.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:06:48.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:06:48.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:06:48.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:06:48.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:06:48.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:06:48.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:48.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:48.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:06:48.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:06:48.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:06:48.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:48.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ3 2026-02-17T06:06:48.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 3 % 2 2026-02-17T06:06:48.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:06:48.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:06:48.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1052: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ3 remove 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ3 remove 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:06:48.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ3 remove 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:06:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ3 remove 2026-02-17T06:06:50.932 INFO:tasks.workunit.client.0.trial074.stdout:remove #3:f2a5b2a4:::ROBJ3:head# 2026-02-17T06:06:51.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:06:51.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:06:51.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:06:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:06:51.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:06:51.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:06:51.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:06:51.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:06:51.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:06:51.496 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:51.487+0000 7f524eb438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:51.497 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:51.487+0000 7f524eb438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:51.498 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:51.487+0000 7f524eb438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:06:51.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:51.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:51.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:52.728 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:52.719+0000 7f524eb438c0 -1 Falling back to public interface 2026-02-17T06:06:52.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:52.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:52.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:06:52.777 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:06:52.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:52.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:52.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:06:53.928 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:06:53.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:53.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:54.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:55.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:55.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:55.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:06:55.078 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:06:55.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:55.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:55.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:06:55.701 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:06:55.691+0000 7f524eb438c0 -1 osd.1 73 log_to_monitors true 2026-02-17T06:06:56.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:06:56.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:06:56.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:06:56.234 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:06:56.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:06:56.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:06:56.383 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [66,73) [v2:127.0.0.1:6810/1970975343,v1:127.0.0.1:6811/1970975343] [v2:127.0.0.1:6812/1970975343,v1:127.0.0.1:6813/1970975343] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:06:56.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:06:56.385 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:06:56.385 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:06:56.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:06:56.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:06:56.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:06:56.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:06:56.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:06:56.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:06:56.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:06:56.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:06:56.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:06:56.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:06:56.655 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:06:56.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:06:56.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:56.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:06:56.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645317 2026-02-17T06:06:56.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645317 2026-02-17T06:06:56.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645317' 2026-02-17T06:06:56.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:06:56.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:06:56.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481794 2026-02-17T06:06:56.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481794 2026-02-17T06:06:56.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645317 1-330712481794' 2026-02-17T06:06:56.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:56.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-309237645317 2026-02-17T06:06:56.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:56.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:06:56.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-309237645317 2026-02-17T06:06:56.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:56.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645317 2026-02-17T06:06:56.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 309237645317' 2026-02-17T06:06:56.805 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 309237645317 2026-02-17T06:06:56.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:56.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645315 -lt 309237645317 2026-02-17T06:06:56.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:06:57.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:06:57.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:06:58.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645317 -lt 309237645317 2026-02-17T06:06:58.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:06:58.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-330712481794 2026-02-17T06:06:58.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:06:58.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:06:58.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-330712481794 2026-02-17T06:06:58.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:06:58.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481794 2026-02-17T06:06:58.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 330712481794' 2026-02-17T06:06:58.122 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 330712481794 2026-02-17T06:06:58.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:06:58.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481794 -lt 330712481794 2026-02-17T06:06:58.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:06:58.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:58.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:58.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:06:58.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:06:58.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:06:58.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:06:58.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:06:58.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:06:58.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:06:58.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:06:58.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:06:58.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:06:58.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:06:58.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:06:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:06:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:06:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:06:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:06:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ4 2026-02-17T06:06:58.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 4 % 2 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1057: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:06:58.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:06:58.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:06:58.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:06:58.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:06:58.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:06:58.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:06:58.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:07:00.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:07:01.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:07:01.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:07:01.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:07:01.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:07:01.262 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:07:01.262 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:07:01.262 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:07:01.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:07:01.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:07:01.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:07:01.264 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:07:01.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:07:01.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:07:01.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:07:01.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:07:01.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:07:01.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:07:01.287 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:01.279+0000 7fcadc19f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:01.288 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:01.279+0000 7fcadc19f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:01.289 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:01.279+0000 7fcadc19f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:01.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:07:01.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:07:01.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:07:01.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:07:01.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:07:01.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:01.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:07:01.415 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:07:01.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:01.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:01.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:02.524 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:02.515+0000 7fcadc19f8c0 -1 Falling back to public interface 2026-02-17T06:07:02.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:02.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:02.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:07:02.568 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:07:02.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:02.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:03.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:03.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:03.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:07:03.717 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:07:03.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:03.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:03.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:04.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:04.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:04.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:07:04.875 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:07:04.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:04.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:05.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:05.500 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:05.491+0000 7fcadc19f8c0 -1 osd.0 78 log_to_monitors true 2026-02-17T06:07:06.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:06.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:06.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:07:06.027 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:07:06.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:06.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [72,78) [v2:127.0.0.1:6802/3949669786,v1:127.0.0.1:6803/3949669786] [v2:127.0.0.1:6804/3949669786,v1:127.0.0.1:6805/3949669786] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:07:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:07:06.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:07:06.179 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:07:06.179 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:07:06.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:07:06.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:07:06.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:07:06.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:07:06.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:07:06.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:07:06.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:07:06.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:07:06.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:07:06.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:07:06.447 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:07:06.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:07:06.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:06.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:07:06.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318274 2026-02-17T06:07:06.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318274 2026-02-17T06:07:06.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274' 2026-02-17T06:07:06.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:06.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:07:06.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481797 2026-02-17T06:07:06.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481797 2026-02-17T06:07:06.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274 1-330712481797' 2026-02-17T06:07:06.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:06.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318274 2026-02-17T06:07:06.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:06.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:07:06.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318274 2026-02-17T06:07:06.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:06.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318274 2026-02-17T06:07:06.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318274' 2026-02-17T06:07:06.594 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 352187318274 2026-02-17T06:07:06.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:06.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 352187318274 2026-02-17T06:07:06.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:07:07.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:07:07.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:07.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318274 -lt 352187318274 2026-02-17T06:07:07.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:07.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-330712481797 2026-02-17T06:07:07.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:07.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:07:07.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-330712481797 2026-02-17T06:07:07.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:07.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481797 2026-02-17T06:07:07.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 330712481797' 2026-02-17T06:07:07.913 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 330712481797 2026-02-17T06:07:07.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:07:08.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481797 -lt 330712481797 2026-02-17T06:07:08.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:07:08.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:08.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:08.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:07:08.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:07:08.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:07:08.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:07:08.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:07:08.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:07:08.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:07:08.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:07:08.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:07:08.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:07:08.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:08.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:08.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:07:08.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:07:08.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:07:08.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:07:08.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ5 2026-02-17T06:07:08.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 5 % 2 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1062: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:07:08.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:07:08.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:07:09.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:07:11.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:07:11.071 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:07:11.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:11.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:11.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:07:11.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:07:11.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:07:11.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:07:11.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:07:11.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:07:11.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:07:11.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:07:11.095 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:11.087+0000 7f9d10d608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:11.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:11.087+0000 7f9d10d608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:11.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:11.087+0000 7f9d10d608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:07:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:07:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:07:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:07:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:07:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:11.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:07:11.225 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:07:11.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:11.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:11.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:12.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:12.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:12.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:07:12.379 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:07:12.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:12.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:12.392 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:12.383+0000 7f9d10d608c0 -1 Falling back to public interface 2026-02-17T06:07:12.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:13.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:13.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:13.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:07:13.531 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:07:13.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:13.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:13.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:14.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:14.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:14.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:07:14.684 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:07:14.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:14.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:14.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:15.380 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:15.371+0000 7f9d10d608c0 -1 osd.1 84 log_to_monitors true 2026-02-17T06:07:15.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:15.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:15.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:07:15.833 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:07:15.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:15.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:15.983 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 88 up_thru 88 down_at 85 last_clean_interval [77,84) [v2:127.0.0.1:6810/3725883518,v1:127.0.0.1:6811/3725883518] [v2:127.0.0.1:6812/3725883518,v1:127.0.0.1:6813/3725883518] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:07:15.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:07:15.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:07:15.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:07:15.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:07:15.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:07:15.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:07:15.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:07:15.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:07:15.985 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:07:15.985 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:07:15.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:07:15.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:07:15.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:07:16.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:07:16.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:07:16.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:07:16.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:07:16.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:07:16.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:07:16.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:07:16.252 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:07:16.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:07:16.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:16.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:07:16.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318277 2026-02-17T06:07:16.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318277 2026-02-17T06:07:16.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277' 2026-02-17T06:07:16.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:16.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:07:16.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122050 2026-02-17T06:07:16.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122050 2026-02-17T06:07:16.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277 1-377957122050' 2026-02-17T06:07:16.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:16.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318277 2026-02-17T06:07:16.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:16.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:07:16.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318277 2026-02-17T06:07:16.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:16.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318277 2026-02-17T06:07:16.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318277' 2026-02-17T06:07:16.401 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 352187318277 2026-02-17T06:07:16.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:16.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318276 -lt 352187318277 2026-02-17T06:07:16.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:07:17.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:07:17.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:17.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318276 -lt 352187318277 2026-02-17T06:07:17.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:07:18.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:07:18.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:18.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318277 -lt 352187318277 2026-02-17T06:07:18.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:18.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122050 2026-02-17T06:07:18.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:18.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:07:18.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122050 2026-02-17T06:07:18.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:18.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122050 2026-02-17T06:07:18.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122050' 2026-02-17T06:07:18.870 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 377957122050 2026-02-17T06:07:18.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:07:19.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122050 -lt 377957122050 2026-02-17T06:07:19.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:07:19.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:19.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:07:19.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:07:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:07:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:07:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:07:19.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:07:19.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:07:19.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:07:19.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:07:19.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:07:19.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:19.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:19.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:07:19.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:07:19.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:07:19.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:07:19.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ6 2026-02-17T06:07:19.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 6 % 2 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: echo extra 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1068: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:07:19.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:07:19.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:07:19.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:07:19.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:07:20.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:07:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T06:07:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:07:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:07:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:07:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:07:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:07:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T06:07:22.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:07:22.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:07:22.043 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:07:22.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:07:22.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:07:22.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:07:22.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:07:22.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:07:22.047 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:07:22.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:07:22.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:07:22.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:07:22.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:07:22.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:07:22.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:07:22.067 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:22.059+0000 7f94404568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:22.068 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:22.059+0000 7f94404568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:22.069 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:22.059+0000 7f94404568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:22.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:07:22.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:07:22.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:07:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:07:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:07:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:07:22.199 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:07:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:22.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:23.328 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:23.319+0000 7f94404568c0 -1 Falling back to public interface 2026-02-17T06:07:23.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:23.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:23.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:07:23.348 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:07:23.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:23.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:23.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:24.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:24.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:24.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:07:24.495 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:07:24.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:24.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:24.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:25.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:25.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:25.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:07:25.644 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:07:25.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:25.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:25.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:26.295 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:26.287+0000 7f94404568c0 -1 osd.0 89 log_to_monitors true 2026-02-17T06:07:26.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:26.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:26.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:07:26.793 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:07:26.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:26.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:26.942 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 93 up_thru 93 down_at 90 last_clean_interval [82,89) [v2:127.0.0.1:6802/3015782673,v1:127.0.0.1:6803/3015782673] [v2:127.0.0.1:6804/3015782673,v1:127.0.0.1:6805/3015782673] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:07:26.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:07:26.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:07:26.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:07:26.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:07:26.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:07:26.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:07:26.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:07:26.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:07:26.944 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:07:26.944 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:07:26.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:07:26.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:07:26.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:07:27.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:07:27.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:07:27.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:07:27.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:07:27.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:07:27.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:07:27.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:07:27.212 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:07:27.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:07:27.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:27.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:07:27.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=399431958530 2026-02-17T06:07:27.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 399431958530 2026-02-17T06:07:27.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958530' 2026-02-17T06:07:27.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:27.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:07:27.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122053 2026-02-17T06:07:27.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122053 2026-02-17T06:07:27.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958530 1-377957122053' 2026-02-17T06:07:27.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:27.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-399431958530 2026-02-17T06:07:27.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:27.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:07:27.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-399431958530 2026-02-17T06:07:27.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:27.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=399431958530 2026-02-17T06:07:27.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 399431958530' 2026-02-17T06:07:27.357 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 399431958530 2026-02-17T06:07:27.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:27.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318277 -lt 399431958530 2026-02-17T06:07:27.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:07:28.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:07:28.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:28.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 399431958530 -lt 399431958530 2026-02-17T06:07:28.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:28.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122053 2026-02-17T06:07:28.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:28.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:07:28.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122053 2026-02-17T06:07:28.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:28.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122053 2026-02-17T06:07:28.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122053' 2026-02-17T06:07:28.670 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 377957122053 2026-02-17T06:07:28.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:07:28.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122053 -lt 377957122053 2026-02-17T06:07:28.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:07:28.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:28.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:29.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:07:29.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:07:29.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:07:29.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:07:29.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:07:29.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:07:29.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:07:29.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:07:29.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:07:29.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:07:29.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:29.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:29.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:07:29.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:07:29.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:07:29.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/extra-val 2026-02-17T06:07:29.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:07:29.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ7 2026-02-17T06:07:29.342 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 7 % 2 2026-02-17T06:07:29.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:07:29.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1074: TEST_corrupt_scrub_replicated: echo -n newheader 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1075: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:07:29.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:07:29.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:07:29.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:07:29.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:07:29.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:07:29.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:07:29.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:07:30.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:07:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T06:07:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:07:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:07:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:07:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:07:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:07:30.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:07:31.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:07:31.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:07:31.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:07:31.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:07:31.822 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:07:31.822 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:07:31.822 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:07:31.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:07:31.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:07:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:07:31.824 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:07:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:07:31.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:07:31.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:07:31.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:07:31.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:07:31.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:07:31.847 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:31.839+0000 7f36925598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:31.849 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:31.839+0000 7f36925598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:31.850 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:31.839+0000 7f36925598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:31.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:07:31.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:07:31.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:07:31.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:07:31.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:07:31.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:31.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:07:31.979 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:07:31.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:31.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:32.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:33.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:33.087+0000 7f36925598c0 -1 Falling back to public interface 2026-02-17T06:07:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:07:33.132 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:07:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:33.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:33.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:34.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:34.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:34.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:07:34.282 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:07:34.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:34.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:34.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:35.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:35.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:35.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:07:35.434 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:07:35.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:35.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:35.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:36.092 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:36.083+0000 7f36925598c0 -1 osd.1 95 log_to_monitors true 2026-02-17T06:07:36.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:36.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:36.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:07:36.587 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:07:36.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:36.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:36.738 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 99 up_thru 99 down_at 96 last_clean_interval [88,95) [v2:127.0.0.1:6810/66630597,v1:127.0.0.1:6811/66630597] [v2:127.0.0.1:6812/66630597,v1:127.0.0.1:6813/66630597] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:07:36.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:07:36.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:07:36.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:07:36.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:07:36.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:07:36.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:07:36.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:07:36.739 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:07:36.740 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:07:36.740 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:07:36.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:07:36.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:07:36.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:07:36.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:07:36.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:07:36.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:07:36.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:07:36.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:07:36.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:07:37.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:07:37.003 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:07:37.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:07:37.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:37.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:07:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=399431958533 2026-02-17T06:07:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 399431958533 2026-02-17T06:07:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958533' 2026-02-17T06:07:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:37.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:07:37.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762306 2026-02-17T06:07:37.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762306 2026-02-17T06:07:37.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958533 1-425201762306' 2026-02-17T06:07:37.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:37.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-399431958533 2026-02-17T06:07:37.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:37.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:07:37.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-399431958533 2026-02-17T06:07:37.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=399431958533 2026-02-17T06:07:37.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 399431958533' 2026-02-17T06:07:37.149 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 399431958533 2026-02-17T06:07:37.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:37.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 399431958531 -lt 399431958533 2026-02-17T06:07:37.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:07:38.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:07:38.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:38.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 399431958533 -lt 399431958533 2026-02-17T06:07:38.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:38.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-425201762306 2026-02-17T06:07:38.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:38.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:07:38.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-425201762306 2026-02-17T06:07:38.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:38.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762306 2026-02-17T06:07:38.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 425201762306' 2026-02-17T06:07:38.466 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 425201762306 2026-02-17T06:07:38.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:07:38.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762306 2026-02-17T06:07:38.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:07:38.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:38.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:38.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:07:38.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:07:38.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:07:38.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:07:38.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:07:38.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:07:38.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:07:38.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:07:38.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:07:38.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:07:38.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:38.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:39.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:07:39.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:07:39.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:07:39.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1076: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/hdr 2026-02-17T06:07:39.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:07:39.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ8 2026-02-17T06:07:39.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 8 % 2 2026-02-17T06:07:39.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:07:39.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:07:39.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1080: TEST_corrupt_scrub_replicated: rados --pool csr_pool setxattr ROBJ8 key1-ROBJ8 val1-ROBJ8 2026-02-17T06:07:39.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1081: TEST_corrupt_scrub_replicated: rados --pool csr_pool setxattr ROBJ8 key2-ROBJ8 val2-ROBJ8 2026-02-17T06:07:39.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1084: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-17T06:07:39.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1085: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T06:07:39.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:07:39.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:07:39.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:07:39.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:07:39.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:07:39.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:07:39.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:07:39.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:07:40.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:07:40.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T06:07:40.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:07:40.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:07:40.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:07:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:07:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:07:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:07:41.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:07:41.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:07:41.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:07:41.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:07:41.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:07:41.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:07:41.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:07:41.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:07:41.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:07:41.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:07:41.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:07:41.667 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:07:41.667 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:07:41.667 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:07:41.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:07:41.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:07:41.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:07:41.669 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:07:41.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:07:41.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:07:41.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:07:41.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:07:41.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:07:41.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:07:41.691 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:41.683+0000 7fc5310488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:41.692 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:41.683+0000 7fc5310488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:41.693 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:41.683+0000 7fc5310488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:41.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:07:41.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:07:41.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:07:41.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:07:41.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:07:41.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:41.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:07:41.821 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:07:41.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:41.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:41.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:42.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:42.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:42.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:07:42.974 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:07:42.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:42.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:43.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:43.163 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:43.155+0000 7fc5310488c0 -1 Falling back to public interface 2026-02-17T06:07:44.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:44.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:44.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:07:44.126 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:07:44.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:44.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:44.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:45.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:45.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:45.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:07:45.278 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:07:45.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:45.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:45.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:46.151 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:46.143+0000 7fc5310488c0 -1 osd.0 101 log_to_monitors true 2026-02-17T06:07:46.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:46.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:46.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:07:46.431 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:07:46.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:46.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:07:46.583 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 105 up_thru 105 down_at 102 last_clean_interval [93,101) [v2:127.0.0.1:6802/800251583,v1:127.0.0.1:6803/800251583] [v2:127.0.0.1:6804/800251583,v1:127.0.0.1:6805/800251583] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:07:46.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:07:46.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:07:46.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:07:46.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:07:46.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:07:46.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:07:46.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:07:46.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:07:46.585 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:07:46.585 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:07:46.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:07:46.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:07:46.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:07:46.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:07:46.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:07:46.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:07:46.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:07:46.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:07:46.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:07:46.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:07:46.851 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:07:46.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:07:46.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:46.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:07:46.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566082 2026-02-17T06:07:46.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566082 2026-02-17T06:07:46.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566082' 2026-02-17T06:07:46.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:46.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:07:46.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762309 2026-02-17T06:07:46.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762309 2026-02-17T06:07:46.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566082 1-425201762309' 2026-02-17T06:07:46.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:46.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566082 2026-02-17T06:07:46.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:46.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:07:46.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566082 2026-02-17T06:07:46.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:46.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566082 2026-02-17T06:07:46.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566082' 2026-02-17T06:07:46.998 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 450971566082 2026-02-17T06:07:46.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:47.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 399431958533 -lt 450971566082 2026-02-17T06:07:47.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:07:48.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:07:48.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:48.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566082 -lt 450971566082 2026-02-17T06:07:48.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:48.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-425201762309 2026-02-17T06:07:48.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:48.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:07:48.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-425201762309 2026-02-17T06:07:48.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:48.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762309 2026-02-17T06:07:48.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 425201762309' 2026-02-17T06:07:48.316 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 425201762309 2026-02-17T06:07:48.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:07:48.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762309 -lt 425201762309 2026-02-17T06:07:48.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:07:48.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:48.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:48.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:07:48.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:07:48.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:07:48.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:07:48.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:07:48.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:07:48.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:07:48.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:07:48.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:07:48.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:07:48.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:48.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1086: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:07:48.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:07:48.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:07:48.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:07:48.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:07:48.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:07:48.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:07:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T06:07:51.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:07:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:07:51.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:07:51.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:07:51.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:07:51.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:07:51.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:07:51.482 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:51.475+0000 7fba624538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:51.483 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:51.475+0000 7fba624538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:51.484 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:51.475+0000 7fba624538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:07:51.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:07:51.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:51.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:52.744 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:52.735+0000 7fba624538c0 -1 Falling back to public interface 2026-02-17T06:07:52.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:52.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:52.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:07:52.771 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:07:52.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:52.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:52.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:53.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:53.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:53.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:07:53.921 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:07:53.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:53.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:54.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:55.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:55.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:55.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:07:55.073 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:07:55.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:55.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:55.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:07:55.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:07:55.707+0000 7fba624538c0 -1 osd.1 106 log_to_monitors true 2026-02-17T06:07:56.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:07:56.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:07:56.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:07:56.225 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:07:56.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:07:56.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:07:56.377 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 110 up_thru 110 down_at 107 last_clean_interval [99,106) [v2:127.0.0.1:6810/91769875,v1:127.0.0.1:6811/91769875] [v2:127.0.0.1:6812/91769875,v1:127.0.0.1:6813/91769875] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:07:56.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:07:56.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:07:56.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:07:56.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:07:56.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:07:56.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:07:56.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:07:56.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:07:56.379 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:07:56.379 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:07:56.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:07:56.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:07:56.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:07:56.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:07:56.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:07:56.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:07:56.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:07:56.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:07:56.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:07:56.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:07:56.643 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:07:56.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:07:56.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:56.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:07:56.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566085 2026-02-17T06:07:56.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566085 2026-02-17T06:07:56.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566085' 2026-02-17T06:07:56.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:07:56.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:07:56.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402562 2026-02-17T06:07:56.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402562 2026-02-17T06:07:56.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566085 1-472446402562' 2026-02-17T06:07:56.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:56.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566085 2026-02-17T06:07:56.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:56.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:07:56.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566085 2026-02-17T06:07:56.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:56.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566085 2026-02-17T06:07:56.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566085' 2026-02-17T06:07:56.790 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 450971566085 2026-02-17T06:07:56.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:56.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566083 -lt 450971566085 2026-02-17T06:07:56.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:07:57.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:07:57.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:07:58.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566085 -lt 450971566085 2026-02-17T06:07:58.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:07:58.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-472446402562 2026-02-17T06:07:58.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:07:58.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:07:58.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-472446402562 2026-02-17T06:07:58.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:07:58.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402562 2026-02-17T06:07:58.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 472446402562' 2026-02-17T06:07:58.097 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 472446402562 2026-02-17T06:07:58.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:07:58.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402562 -lt 472446402562 2026-02-17T06:07:58.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:07:58.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:58.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:58.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:07:58.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:07:58.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:07:58.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:07:58.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:07:58.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:07:58.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:07:58.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:07:58.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:07:58.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:07:58.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:07:58.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:07:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:07:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:07:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:07:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1087: TEST_corrupt_scrub_replicated: echo -n val3-ROBJ8 2026-02-17T06:07:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T06:07:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:07:58.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:07:58.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:07:58.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:08:00.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:08:00.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T06:08:00.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:08:00.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:08:00.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:08:00.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:08:00.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:00.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T06:08:01.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:08:01.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:08:01.254 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:08:01.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:08:01.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:08:01.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:08:01.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:08:01.256 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:08:01.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:08:01.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:08:01.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:08:01.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:08:01.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:08:01.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:08:01.270 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:01.263+0000 7fc04a0ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:01.271 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:01.263+0000 7fc04a0ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:01.272 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:01.263+0000 7fc04a0ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:01.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:08:01.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:01.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:01.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:02.500 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:02.491+0000 7fc04a0ae8c0 -1 Falling back to public interface 2026-02-17T06:08:02.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:02.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:02.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:08:02.561 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:08:02.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:02.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:02.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:03.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:03.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:03.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:08:03.708 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:08:03.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:03.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:03.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:04.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:04.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:04.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:08:04.860 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:08:04.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:04.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:05.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:05.514 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:05.507+0000 7fc04a0ae8c0 -1 osd.1 112 log_to_monitors true 2026-02-17T06:08:06.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:06.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:06.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:08:06.011 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:08:06.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:06.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:06.158 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 116 up_thru 116 down_at 113 last_clean_interval [110,112) [v2:127.0.0.1:6810/4168871632,v1:127.0.0.1:6811/4168871632] [v2:127.0.0.1:6812/4168871632,v1:127.0.0.1:6813/4168871632] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:08:06.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:08:06.160 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:08:06.160 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:08:06.160 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:08:06.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:08:06.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:08:06.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:08:06.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:08:06.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:08:06.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:08:06.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:08:06.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:08:06.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:08:06.425 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:08:06.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:08:06.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:06.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:08:06.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566088 2026-02-17T06:08:06.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566088 2026-02-17T06:08:06.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566088' 2026-02-17T06:08:06.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:06.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:08:06.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=498216206338 2026-02-17T06:08:06.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 498216206338 2026-02-17T06:08:06.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566088 1-498216206338' 2026-02-17T06:08:06.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:06.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566088 2026-02-17T06:08:06.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:06.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:08:06.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566088 2026-02-17T06:08:06.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:06.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566088 2026-02-17T06:08:06.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566088' 2026-02-17T06:08:06.572 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 450971566088 2026-02-17T06:08:06.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:06.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566086 -lt 450971566088 2026-02-17T06:08:06.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:07.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:08:07.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:07.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566088 -lt 450971566088 2026-02-17T06:08:07.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:07.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-498216206338 2026-02-17T06:08:07.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:07.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:08:07.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-498216206338 2026-02-17T06:08:07.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:07.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=498216206338 2026-02-17T06:08:07.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 498216206338' 2026-02-17T06:08:07.886 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 498216206338 2026-02-17T06:08:07.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:08:08.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 498216206338 -lt 498216206338 2026-02-17T06:08:08.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:08:08.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:08.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:08:08.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:08:08.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:08:08.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:08:08.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:08.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:08.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:08:08.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:08:08.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:08:08.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-17T06:08:08.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:08.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-17T06:08:08.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 9 % 2 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1093: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:08:08.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:08:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:08:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:08:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-17T06:08:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:08:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:08:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:08:09.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:08:09.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:09.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 get-attr _ 2026-02-17T06:08:10.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:08:10.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:08:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:08:10.714 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:08:10.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:10.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:10.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:08:10.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:08:10.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:08:10.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:08:10.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:08:10.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:08:10.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:08:10.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:08:10.729 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:10.719+0000 7f31a90a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:10.731 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:10.723+0000 7f31a90a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:10.732 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:10.723+0000 7f31a90a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:10.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:10.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:11.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:12.008 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:11.999+0000 7f31a90a58c0 -1 Falling back to public interface 2026-02-17T06:08:12.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:12.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:12.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:08:12.025 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:08:12.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:12.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:12.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:13.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:13.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:13.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:08:13.176 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:08:13.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:13.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:13.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:14.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:14.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:14.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:08:14.328 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:08:14.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:14.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:14.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:15.003 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:14.995+0000 7f31a90a58c0 -1 osd.1 117 log_to_monitors true 2026-02-17T06:08:15.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:15.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:15.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:08:15.476 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:08:15.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:15.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:15.623 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 121 up_thru 121 down_at 118 last_clean_interval [116,117) [v2:127.0.0.1:6810/2581300973,v1:127.0.0.1:6811/2581300973] [v2:127.0.0.1:6812/2581300973,v1:127.0.0.1:6813/2581300973] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:08:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:08:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:08:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:08:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:08:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:08:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:08:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:08:15.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:08:15.625 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:08:15.625 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:08:15.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:08:15.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:08:15.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:08:15.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:08:15.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:08:15.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:08:15.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:08:15.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:08:15.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:08:15.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:08:15.883 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:08:15.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:08:15.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:15.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:08:15.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566090 2026-02-17T06:08:15.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566090 2026-02-17T06:08:15.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566090' 2026-02-17T06:08:15.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:15.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:08:16.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=519691042818 2026-02-17T06:08:16.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 519691042818 2026-02-17T06:08:16.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566090 1-519691042818' 2026-02-17T06:08:16.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:16.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566090 2026-02-17T06:08:16.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:16.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:08:16.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566090 2026-02-17T06:08:16.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:16.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566090 2026-02-17T06:08:16.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566090' 2026-02-17T06:08:16.027 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 450971566090 2026-02-17T06:08:16.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:16.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566089 -lt 450971566090 2026-02-17T06:08:16.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:17.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:08:17.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:17.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566089 -lt 450971566090 2026-02-17T06:08:17.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:18.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:08:18.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:18.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566091 -lt 450971566090 2026-02-17T06:08:18.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:18.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-519691042818 2026-02-17T06:08:18.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:18.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:08:18.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-519691042818 2026-02-17T06:08:18.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:18.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=519691042818 2026-02-17T06:08:18.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 519691042818' 2026-02-17T06:08:18.503 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 519691042818 2026-02-17T06:08:18.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:08:18.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 519691042818 -lt 519691042818 2026-02-17T06:08:18.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:08:18.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:18.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:18.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:08:18.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:08:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:08:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:08:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:08:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:08:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:08:18.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:08:18.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:08:18.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:08:18.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:18.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:19.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:08:19.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:08:19.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:08:19.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: echo -n D 2026-02-17T06:08:19.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1095: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/change 2026-02-17T06:08:19.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1096: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:08:19.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:08:19.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:08:19.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:08:19.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:08:19.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:08:20.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:08:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:08:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:08:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:08:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:08:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:08:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:08:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:08:21.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:08:21.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:08:21.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:08:21.676 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:08:21.676 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:08:21.677 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:08:21.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:21.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:21.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:08:21.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:08:21.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:08:21.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:08:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:08:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:08:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:08:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:08:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:08:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:08:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:08:21.679 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:08:21.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:08:21.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:08:21.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:08:21.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:08:21.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:08:21.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:08:21.702 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:21.695+0000 7f9fdc2578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:21.703 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:21.695+0000 7f9fdc2578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:21.704 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:21.695+0000 7f9fdc2578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:21.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:08:21.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:08:21.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:08:21.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:08:21.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:08:21.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:21.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:08:21.834 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:08:21.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:21.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:21.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:22.947 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:22.939+0000 7f9fdc2578c0 -1 Falling back to public interface 2026-02-17T06:08:22.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:22.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:22.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:08:22.983 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:08:22.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:22.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:23.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:24.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:24.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:24.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:08:24.134 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:08:24.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:24.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:24.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:25.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:25.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:25.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:08:25.285 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:08:25.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:25.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:25.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:25.898 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:25.887+0000 7f9fdc2578c0 -1 osd.1 123 log_to_monitors true 2026-02-17T06:08:26.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:26.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:26.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:08:26.435 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:08:26.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:26.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:26.582 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 127 up_thru 127 down_at 124 last_clean_interval [121,123) [v2:127.0.0.1:6810/3561083645,v1:127.0.0.1:6811/3561083645] [v2:127.0.0.1:6812/3561083645,v1:127.0.0.1:6813/3561083645] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:08:26.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:08:26.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:08:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:08:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:08:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:08:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:08:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:08:26.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:08:26.584 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:08:26.584 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:08:26.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:08:26.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:08:26.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:08:26.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:08:26.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:08:26.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:08:26.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:08:26.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:08:26.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:08:26.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:08:26.846 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:08:26.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:08:26.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:26.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:08:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566094 2026-02-17T06:08:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566094 2026-02-17T06:08:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566094' 2026-02-17T06:08:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:26.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:08:26.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=545460846594 2026-02-17T06:08:26.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 545460846594 2026-02-17T06:08:26.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566094 1-545460846594' 2026-02-17T06:08:26.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:26.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566094 2026-02-17T06:08:26.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:26.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:08:26.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566094 2026-02-17T06:08:26.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:26.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566094 2026-02-17T06:08:26.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566094' 2026-02-17T06:08:26.994 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 450971566094 2026-02-17T06:08:26.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:27.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566092 -lt 450971566094 2026-02-17T06:08:27.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:28.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:08:28.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:28.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566094 -lt 450971566094 2026-02-17T06:08:28.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:28.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-545460846594 2026-02-17T06:08:28.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:28.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:08:28.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-545460846594 2026-02-17T06:08:28.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:28.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=545460846594 2026-02-17T06:08:28.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 545460846594' 2026-02-17T06:08:28.314 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 545460846594 2026-02-17T06:08:28.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:08:28.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 545460846594 -lt 545460846594 2026-02-17T06:08:28.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:08:28.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:28.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:08:28.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:08:28.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:08:28.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:08:28.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:08:28.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:08:28.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:08:28.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:08:28.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:08:28.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:08:28.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:28.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:28.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:08:28.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:08:28.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:08:28.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1097: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi td/osd-scrub-repair/change 2026-02-17T06:08:28.983 INFO:tasks.workunit.client.0.trial074.stderr:rm: cannot remove 'td/osd-scrub-repair/oi': No such file or directory 2026-02-17T06:08:28.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:28.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-17T06:08:28.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 10 % 2 2026-02-17T06:08:28.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:08:28.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:28.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:28.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ11 2026-02-17T06:08:28.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 11 % 2 2026-02-17T06:08:28.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:08:28.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:28.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:28.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ12 2026-02-17T06:08:28.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 12 % 2 2026-02-17T06:08:28.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:08:28.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:28.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:28.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ13 2026-02-17T06:08:28.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 13 % 2 2026-02-17T06:08:28.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:08:28.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:28.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:28.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ14 2026-02-17T06:08:28.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 14 % 2 2026-02-17T06:08:28.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:08:28.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:28.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1106: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-17T06:08:28.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1107: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T06:08:28.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:08:28.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:08:28.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:08:28.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:08:30.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:08:30.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T06:08:30.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:08:30.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:08:30.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:08:30.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:08:30.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:08:30.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T06:08:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:08:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:08:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:08:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:08:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:08:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:08:31.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:08:31.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:08:31.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:08:31.475 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:08:31.475 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:08:31.475 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:08:31.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:08:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:08:31.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:08:31.498 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:31.491+0000 7f58f4ed68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:31.499 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:31.491+0000 7f58f4ed68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:31.500 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:31.491+0000 7f58f4ed68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:31.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:08:31.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:31.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:08:31.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:32.759 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:32.751+0000 7f58f4ed68c0 -1 Falling back to public interface 2026-02-17T06:08:32.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:32.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:32.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:08:32.783 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:08:32.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:32.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:08:32.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:33.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:33.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:33.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:08:33.932 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:08:33.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:33.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:08:34.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:35.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:35.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:35.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:08:35.084 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:08:35.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:35.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:08:35.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:35.745 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:35.735+0000 7f58f4ed68c0 -1 osd.0 128 log_to_monitors true 2026-02-17T06:08:36.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:36.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:36.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:08:36.240 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:08:36.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:36.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:08:36.385 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 132 up_thru 132 down_at 129 last_clean_interval [105,128) [v2:127.0.0.1:6802/1343873917,v1:127.0.0.1:6803/1343873917] [v2:127.0.0.1:6804/1343873917,v1:127.0.0.1:6805/1343873917] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:08:36.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:08:36.387 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:08:36.387 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:08:36.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:08:36.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:08:36.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:08:36.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:08:36.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:08:36.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:08:36.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:08:36.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:08:36.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:08:36.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:08:36.662 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:08:36.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:08:36.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:36.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:08:36.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=566935683074 2026-02-17T06:08:36.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 566935683074 2026-02-17T06:08:36.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-566935683074' 2026-02-17T06:08:36.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:36.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:08:36.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=545460846597 2026-02-17T06:08:36.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 545460846597 2026-02-17T06:08:36.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-566935683074 1-545460846597' 2026-02-17T06:08:36.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:36.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-566935683074 2026-02-17T06:08:36.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:36.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:08:36.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-566935683074 2026-02-17T06:08:36.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:36.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=566935683074 2026-02-17T06:08:36.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 566935683074' 2026-02-17T06:08:36.808 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 566935683074 2026-02-17T06:08:36.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:36.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566094 -lt 566935683074 2026-02-17T06:08:36.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:37.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:08:37.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:38.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 566935683074 -lt 566935683074 2026-02-17T06:08:38.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:38.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-545460846597 2026-02-17T06:08:38.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:38.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:08:38.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-545460846597 2026-02-17T06:08:38.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:38.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=545460846597 2026-02-17T06:08:38.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 545460846597' 2026-02-17T06:08:38.125 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 545460846597 2026-02-17T06:08:38.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:08:38.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 545460846597 -lt 545460846597 2026-02-17T06:08:38.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:08:38.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:38.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:38.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:08:38.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:08:38.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:08:38.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:08:38.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:08:38.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:08:38.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:08:38.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:08:38.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:08:38.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:08:38.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:38.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1108: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:08:38.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:08:38.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:08:38.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:08:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:08:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-17T06:08:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:08:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:08:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:08:40.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:08:40.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:40.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ14 rm-attr _ 2026-02-17T06:08:41.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:08:41.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:08:41.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:08:41.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:08:41.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:08:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:08:41.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:08:41.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:08:41.306 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:41.299+0000 7f9c2f07a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:41.307 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:41.299+0000 7f9c2f07a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:41.309 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:41.299+0000 7f9c2f07a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:41.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:41.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:41.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:42.536 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:42.527+0000 7f9c2f07a8c0 -1 Falling back to public interface 2026-02-17T06:08:42.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:42.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:42.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:08:42.585 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:08:42.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:42.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:42.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:43.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:43.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:43.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:08:43.739 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:08:43.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:43.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:43.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:44.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:44.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:44.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:08:44.891 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:08:44.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:44.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:45.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:45.534 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:45.527+0000 7f9c2f07a8c0 -1 osd.1 133 log_to_monitors true 2026-02-17T06:08:46.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:46.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:46.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:08:46.045 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:08:46.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:46.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:46.196 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 137 up_thru 137 down_at 134 last_clean_interval [127,133) [v2:127.0.0.1:6810/1091322669,v1:127.0.0.1:6811/1091322669] [v2:127.0.0.1:6812/1091322669,v1:127.0.0.1:6813/1091322669] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:08:46.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:08:46.198 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:08:46.198 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:08:46.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:08:46.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:08:46.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:08:46.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:08:46.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:08:46.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:08:46.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:08:46.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:08:46.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:08:46.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:08:46.469 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:08:46.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:08:46.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:46.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:08:46.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=566935683077 2026-02-17T06:08:46.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 566935683077 2026-02-17T06:08:46.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-566935683077' 2026-02-17T06:08:46.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:46.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:08:46.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=588410519554 2026-02-17T06:08:46.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 588410519554 2026-02-17T06:08:46.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-566935683077 1-588410519554' 2026-02-17T06:08:46.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:46.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-566935683077 2026-02-17T06:08:46.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:46.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:08:46.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-566935683077 2026-02-17T06:08:46.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:46.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=566935683077 2026-02-17T06:08:46.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 566935683077' 2026-02-17T06:08:46.618 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 566935683077 2026-02-17T06:08:46.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:46.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 566935683075 -lt 566935683077 2026-02-17T06:08:46.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:47.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:08:47.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:47.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 566935683077 -lt 566935683077 2026-02-17T06:08:47.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:47.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-588410519554 2026-02-17T06:08:47.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:47.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:08:47.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-588410519554 2026-02-17T06:08:47.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:47.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=588410519554 2026-02-17T06:08:47.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 588410519554' 2026-02-17T06:08:47.929 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 588410519554 2026-02-17T06:08:47.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:08:48.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 588410519554 -lt 588410519554 2026-02-17T06:08:48.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:08:48.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:48.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:48.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:08:48.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:08:48.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:08:48.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:08:48.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:08:48.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:08:48.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:08:48.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:08:48.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:08:48.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:08:48.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:48.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:48.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:08:48.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:08:48.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:08:48.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1109: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/bad-val 2026-02-17T06:08:48.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:48.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ15 2026-02-17T06:08:48.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 15 % 2 2026-02-17T06:08:48.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1113: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:08:48.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:08:48.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:08:48.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:08:48.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:08:48.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:08:48.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:49.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ15 rm-attr _ 2026-02-17T06:08:50.558 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:08:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:08:51.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:08:51.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:08:51.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:08:51.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:08:51.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:08:51.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:08:51.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:08:51.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:08:51.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:08:51.123 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:51.115+0000 7f292d1b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:51.124 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:51.115+0000 7f292d1b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:51.125 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:51.119+0000 7f292d1b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:08:51.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:51.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:51.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:52.376 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:52.367+0000 7f292d1b28c0 -1 Falling back to public interface 2026-02-17T06:08:52.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:52.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:52.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:08:52.405 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:08:52.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:52.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:52.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:53.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:53.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:53.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:08:53.558 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:08:53.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:53.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:53.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:54.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:54.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:54.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:08:54.714 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:08:54.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:54.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:54.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:08:55.359 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:08:55.351+0000 7f292d1b28c0 -1 osd.1 139 log_to_monitors true 2026-02-17T06:08:55.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:08:55.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:08:55.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:08:55.870 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:08:55.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:08:55.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 143 up_thru 143 down_at 140 last_clean_interval [137,139) [v2:127.0.0.1:6810/3203461730,v1:127.0.0.1:6811/3203461730] [v2:127.0.0.1:6812/3203461730,v1:127.0.0.1:6813/3203461730] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:08:56.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:08:56.020 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:08:56.020 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:08:56.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:08:56.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:08:56.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:08:56.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:08:56.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:08:56.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:08:56.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:08:56.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:08:56.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:08:56.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:08:56.288 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:08:56.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:08:56.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:56.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:08:56.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=566935683080 2026-02-17T06:08:56.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 566935683080 2026-02-17T06:08:56.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-566935683080' 2026-02-17T06:08:56.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:08:56.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:08:56.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=614180323330 2026-02-17T06:08:56.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 614180323330 2026-02-17T06:08:56.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-566935683080 1-614180323330' 2026-02-17T06:08:56.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:56.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-566935683080 2026-02-17T06:08:56.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:56.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:08:56.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-566935683080 2026-02-17T06:08:56.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:56.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=566935683080 2026-02-17T06:08:56.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 566935683080' 2026-02-17T06:08:56.432 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 566935683080 2026-02-17T06:08:56.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:56.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 566935683078 -lt 566935683080 2026-02-17T06:08:56.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:57.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:08:57.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:57.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 566935683078 -lt 566935683080 2026-02-17T06:08:57.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:08:58.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:08:58.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:08:58.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 566935683080 -lt 566935683080 2026-02-17T06:08:58.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:08:58.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-614180323330 2026-02-17T06:08:58.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:08:58.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:08:58.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-614180323330 2026-02-17T06:08:58.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:08:58.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=614180323330 2026-02-17T06:08:58.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 614180323330' 2026-02-17T06:08:58.904 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 614180323330 2026-02-17T06:08:58.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:08:59.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 614180323330 -lt 614180323330 2026-02-17T06:08:59.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:08:59.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:59.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:59.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:08:59.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:08:59.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:08:59.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:08:59.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:08:59.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:08:59.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:08:59.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:08:59.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:08:59.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:08:59.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:08:59.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:08:59.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:08:59.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:08:59.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:08:59.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:08:59.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ16 2026-02-17T06:08:59.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 16 % 2 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1117: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:08:59.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:08:59.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:08:59.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:08:59.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:08:59.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:08:59.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:09:00.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:09:00.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-17T06:09:00.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:09:00.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:09:00.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:09:00.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:09:00.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:09:00.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ16 rm-attr snapset 2026-02-17T06:09:01.527 INFO:tasks.workunit.client.0.trial074.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T06:09:02.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:09:02.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:09:02.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:09:02.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:09:02.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:09:02.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:09:02.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:09:02.068 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:09:02.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:09:02.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:09:02.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:09:02.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:09:02.090 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:02.083+0000 7f12b441e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:02.091 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:02.083+0000 7f12b441e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:02.092 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:02.083+0000 7f12b441e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:09:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:09:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:09:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:09:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:09:02.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:02.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:09:02.221 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:09:02.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:02.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:02.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:03.303 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:03.295+0000 7f12b441e8c0 -1 Falling back to public interface 2026-02-17T06:09:03.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:03.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:03.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:09:03.375 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:09:03.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:03.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:03.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:04.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:04.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:04.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:09:04.527 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:09:04.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:04.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:04.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:05.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:05.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:05.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:09:05.677 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:09:05.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:05.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:05.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:06.347 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:06.339+0000 7f12b441e8c0 -1 osd.0 145 log_to_monitors true 2026-02-17T06:09:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:09:06.831 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:09:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:06.982 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 149 up_thru 149 down_at 146 last_clean_interval [132,145) [v2:127.0.0.1:6802/135861435,v1:127.0.0.1:6803/135861435] [v2:127.0.0.1:6804/135861435,v1:127.0.0.1:6805/135861435] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:09:06.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:09:06.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:09:06.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:09:06.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:09:06.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:09:06.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:09:06.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:09:06.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:09:06.984 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:09:06.984 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:09:06.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:09:06.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:09:06.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:09:07.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:09:07.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:09:07.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:09:07.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:09:07.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:09:07.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:09:07.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:09:07.248 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:09:07.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:09:07.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:07.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:09:07.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=639950127106 2026-02-17T06:09:07.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 639950127106 2026-02-17T06:09:07.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127106' 2026-02-17T06:09:07.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:07.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:09:07.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=614180323333 2026-02-17T06:09:07.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 614180323333 2026-02-17T06:09:07.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127106 1-614180323333' 2026-02-17T06:09:07.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:07.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-639950127106 2026-02-17T06:09:07.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:07.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:09:07.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-639950127106 2026-02-17T06:09:07.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:07.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=639950127106 2026-02-17T06:09:07.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 639950127106' 2026-02-17T06:09:07.396 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 639950127106 2026-02-17T06:09:07.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:07.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 566935683080 -lt 639950127106 2026-02-17T06:09:07.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:09:08.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:09:08.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:08.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 639950127106 -lt 639950127106 2026-02-17T06:09:08.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:08.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-614180323333 2026-02-17T06:09:08.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:08.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:09:08.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-614180323333 2026-02-17T06:09:08.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:08.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=614180323333 2026-02-17T06:09:08.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 614180323333' 2026-02-17T06:09:08.699 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 614180323333 2026-02-17T06:09:08.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:09:08.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 614180323333 -lt 614180323333 2026-02-17T06:09:08.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:09:08.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:08.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:09.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:09:09.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:09:09.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:09:09.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:09:09.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:09:09.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:09:09.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:09:09.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:09:09.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:09:09.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:09:09.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:09.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1118: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1119: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:09:09.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:09:09.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:09:10.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:09:10.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T06:09:10.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:09:10.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:09:10.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:09:10.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:09:10.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:09:10.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T06:09:11.323 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T06:09:11.323 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T06:09:11.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:09:11.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:09:11.861 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:09:11.861 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:09:11.861 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:09:11.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:09:11.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:09:11.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:09:11.863 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:09:11.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:09:11.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:09:11.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:09:11.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:09:11.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:09:11.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:09:11.886 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:11.879+0000 7f9435a868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:11.887 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:11.879+0000 7f9435a868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:11.888 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:11.879+0000 7f9435a868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:12.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:09:12.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:12.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:12.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:13.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:13.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:13.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:09:13.168 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:09:13.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:13.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:13.203 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:13.195+0000 7f9435a868c0 -1 Falling back to public interface 2026-02-17T06:09:13.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:14.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:14.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:14.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:09:14.318 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:09:14.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:14.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:14.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:15.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:15.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:15.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:09:15.469 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:09:15.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:15.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:15.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:16.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:16.235+0000 7f9435a868c0 -1 osd.1 150 log_to_monitors true 2026-02-17T06:09:16.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:16.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:16.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:09:16.618 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:09:16.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:16.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:16.766 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 154 up_thru 154 down_at 151 last_clean_interval [143,150) [v2:127.0.0.1:6810/3129931969,v1:127.0.0.1:6811/3129931969] [v2:127.0.0.1:6812/3129931969,v1:127.0.0.1:6813/3129931969] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:09:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:09:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:09:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:09:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:09:16.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:09:16.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:09:16.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:09:16.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:09:16.768 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:09:16.768 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:09:16.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:09:16.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:09:16.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:09:16.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:09:16.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:09:16.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:09:16.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:09:16.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:09:16.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:09:17.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:09:17.032 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:09:17.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:09:17.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:17.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:09:17.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=639950127109 2026-02-17T06:09:17.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 639950127109 2026-02-17T06:09:17.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127109' 2026-02-17T06:09:17.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:17.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:09:17.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=661424963586 2026-02-17T06:09:17.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 661424963586 2026-02-17T06:09:17.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127109 1-661424963586' 2026-02-17T06:09:17.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:17.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-639950127109 2026-02-17T06:09:17.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:17.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:09:17.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-639950127109 2026-02-17T06:09:17.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:17.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=639950127109 2026-02-17T06:09:17.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 639950127109' 2026-02-17T06:09:17.181 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 639950127109 2026-02-17T06:09:17.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:17.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 639950127107 -lt 639950127109 2026-02-17T06:09:17.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:09:18.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:09:18.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:18.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 639950127109 -lt 639950127109 2026-02-17T06:09:18.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:18.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-661424963586 2026-02-17T06:09:18.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:18.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:09:18.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-661424963586 2026-02-17T06:09:18.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:18.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=661424963586 2026-02-17T06:09:18.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 661424963586' 2026-02-17T06:09:18.500 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 661424963586 2026-02-17T06:09:18.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:09:18.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 661424963586 -lt 661424963586 2026-02-17T06:09:18.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:09:18.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:18.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:09:18.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:09:18.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:09:18.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:09:18.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:18.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:19.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:09:19.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:09:19.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:09:19.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:09:19.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ17 2026-02-17T06:09:19.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 17 % 2 2026-02-17T06:09:19.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:09:19.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:09:19.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1124: TEST_corrupt_scrub_replicated: local payload=ROBJ17 2026-02-17T06:09:19.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1125: TEST_corrupt_scrub_replicated: echo ROBJ17 2026-02-17T06:09:19.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1126: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:19.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:09:19.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:09:19.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:09:19.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:09:19.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:09:19.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:09:20.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:21.120 INFO:tasks.workunit.client.0.trial074.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T06:09:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:09:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:09:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:09:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:09:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:09:21.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:09:21.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:09:21.659 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:09:21.659 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:09:21.659 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:09:21.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:09:21.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:09:21.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:09:21.662 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:09:21.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:09:21.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:09:21.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:09:21.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:09:21.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:09:21.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:09:21.682 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:21.675+0000 7f23e22e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:21.684 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:21.675+0000 7f23e22e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:21.684 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:21.675+0000 7f23e22e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:21.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:21.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:21.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:22.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:22.927+0000 7f23e22e28c0 -1 Falling back to public interface 2026-02-17T06:09:22.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:22.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:22.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:09:22.963 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:09:22.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:22.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:24.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:24.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:24.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:09:24.121 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:09:24.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:24.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:24.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:25.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:25.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:25.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:09:25.275 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:09:25.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:25.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:25.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:26.194 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:26.187+0000 7f23e22e28c0 -1 osd.0 156 log_to_monitors true 2026-02-17T06:09:26.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:26.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:26.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:09:26.430 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:09:26.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:26.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:26.582 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 160 up_thru 160 down_at 157 last_clean_interval [149,156) [v2:127.0.0.1:6802/3587996071,v1:127.0.0.1:6803/3587996071] [v2:127.0.0.1:6804/3587996071,v1:127.0.0.1:6805/3587996071] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:09:26.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:09:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:09:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:09:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:09:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:09:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:09:26.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:09:26.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:09:26.584 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:09:26.584 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:09:26.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:09:26.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:09:26.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:09:26.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:09:26.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:09:26.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:09:26.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:09:26.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:09:26.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:09:26.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:09:26.844 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:09:26.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:09:26.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:26.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:09:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=687194767362 2026-02-17T06:09:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 687194767362 2026-02-17T06:09:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767362' 2026-02-17T06:09:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:26.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:09:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=661424963589 2026-02-17T06:09:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 661424963589 2026-02-17T06:09:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767362 1-661424963589' 2026-02-17T06:09:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:26.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-687194767362 2026-02-17T06:09:26.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:26.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:09:26.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-687194767362 2026-02-17T06:09:26.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:26.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=687194767362 2026-02-17T06:09:26.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 687194767362' 2026-02-17T06:09:26.992 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 687194767362 2026-02-17T06:09:26.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:27.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 639950127109 -lt 687194767362 2026-02-17T06:09:27.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:09:28.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:09:28.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:28.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 687194767362 -lt 687194767362 2026-02-17T06:09:28.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:28.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-661424963589 2026-02-17T06:09:28.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:28.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:09:28.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-661424963589 2026-02-17T06:09:28.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:28.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=661424963589 2026-02-17T06:09:28.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 661424963589' 2026-02-17T06:09:28.302 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 661424963589 2026-02-17T06:09:28.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:09:28.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 661424963589 -lt 661424963589 2026-02-17T06:09:28.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:09:28.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:28.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:28.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:09:28.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:09:28.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:09:28.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:09:28.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:09:28.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:09:28.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:09:28.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:09:28.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:09:28.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:09:28.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:28.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:28.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:09:28.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:09:28.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1127: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:09:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:09:28.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:09:28.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:09:28.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:09:28.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:09:28.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:09:30.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:09:30.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:30.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:09:30.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:09:30.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:09:30.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:09:30.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:09:30.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T06:09:30.911 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T06:09:30.912 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T06:09:31.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:09:31.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:09:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:09:31.450 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:09:31.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:31.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:31.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:09:31.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:09:31.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:09:31.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:09:31.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:09:31.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:09:31.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:09:31.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:09:31.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:09:31.473 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:31.463+0000 7f17992988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:31.474 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:31.467+0000 7f17992988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:31.475 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:31.467+0000 7f17992988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:31.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:09:31.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:09:31.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:09:31.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:09:31.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:09:31.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:31.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:09:31.603 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:09:31.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:31.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:31.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:32.723 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:32.715+0000 7f17992988c0 -1 Falling back to public interface 2026-02-17T06:09:32.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:32.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:32.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:09:32.757 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:09:32.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:32.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:32.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:33.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:33.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:33.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:09:33.910 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:09:33.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:33.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:34.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:35.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:35.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:35.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:09:35.066 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:09:35.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:35.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:35.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:35.726 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:35.719+0000 7f17992988c0 -1 osd.1 161 log_to_monitors true 2026-02-17T06:09:36.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:36.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:36.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:09:36.217 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:09:36.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:36.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 165 up_thru 165 down_at 162 last_clean_interval [154,161) [v2:127.0.0.1:6810/1044839931,v1:127.0.0.1:6811/1044839931] [v2:127.0.0.1:6812/1044839931,v1:127.0.0.1:6813/1044839931] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:09:36.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:09:36.365 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:09:36.365 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:09:36.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:09:36.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:09:36.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:09:36.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:09:36.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:09:36.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:09:36.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:09:36.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:09:36.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:09:36.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:09:36.629 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:09:36.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:09:36.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:36.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:09:36.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=687194767365 2026-02-17T06:09:36.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 687194767365 2026-02-17T06:09:36.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767365' 2026-02-17T06:09:36.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:36.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:09:36.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=708669603842 2026-02-17T06:09:36.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 708669603842 2026-02-17T06:09:36.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767365 1-708669603842' 2026-02-17T06:09:36.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:36.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-687194767365 2026-02-17T06:09:36.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:36.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:09:36.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-687194767365 2026-02-17T06:09:36.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:36.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=687194767365 2026-02-17T06:09:36.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 687194767365' 2026-02-17T06:09:36.774 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 687194767365 2026-02-17T06:09:36.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:36.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 687194767363 -lt 687194767365 2026-02-17T06:09:36.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:09:37.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:09:37.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:38.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 687194767365 -lt 687194767365 2026-02-17T06:09:38.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:38.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-708669603842 2026-02-17T06:09:38.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:38.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:09:38.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-708669603842 2026-02-17T06:09:38.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:38.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=708669603842 2026-02-17T06:09:38.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 708669603842' 2026-02-17T06:09:38.094 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 708669603842 2026-02-17T06:09:38.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:09:38.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 708669603842 -lt 708669603842 2026-02-17T06:09:38.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:09:38.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:38.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:38.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:09:38.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:09:38.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:09:38.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:09:38.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:09:38.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:09:38.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:09:38.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:09:38.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:09:38.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:09:38.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:38.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:38.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:09:38.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:09:38.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:09:38.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:09:38.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ18 2026-02-17T06:09:38.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 18 % 2 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1132: TEST_corrupt_scrub_replicated: local payload=ROBJ18 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1133: TEST_corrupt_scrub_replicated: echo ROBJ18 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1134: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:09:38.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:09:38.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:09:40.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:09:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:09:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:09:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:09:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:09:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:09:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:40.718 INFO:tasks.workunit.client.0.trial074.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T06:09:41.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:09:41.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:09:41.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:09:41.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:09:41.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:09:41.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:09:41.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:09:41.258 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:09:41.259 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:09:41.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:41.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:41.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:09:41.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:09:41.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:09:41.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:09:41.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:09:41.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:09:41.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:09:41.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:09:41.281 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:41.275+0000 7f1101e858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:41.283 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:41.275+0000 7f1101e858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:41.284 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:41.275+0000 7f1101e858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:41.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:41.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:41.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:42.531 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:42.523+0000 7f1101e858c0 -1 Falling back to public interface 2026-02-17T06:09:42.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:42.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:42.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:09:42.568 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:09:42.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:42.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:42.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:43.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:43.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:43.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:09:43.720 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:09:43.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:43.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:43.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:44.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:44.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:44.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:09:44.874 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:09:44.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:44.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:45.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:45.547 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:45.539+0000 7f1101e858c0 -1 osd.0 167 log_to_monitors true 2026-02-17T06:09:46.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:46.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:46.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:09:46.026 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:09:46.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:46.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 171 up_thru 171 down_at 168 last_clean_interval [160,167) [v2:127.0.0.1:6802/2594981525,v1:127.0.0.1:6803/2594981525] [v2:127.0.0.1:6804/2594981525,v1:127.0.0.1:6805/2594981525] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:09:46.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:09:46.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:09:46.176 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:09:46.177 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:09:46.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:09:46.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:09:46.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:09:46.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:09:46.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:09:46.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:09:46.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:09:46.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:09:46.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:09:46.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:09:46.445 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:09:46.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:09:46.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:46.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:09:46.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=734439407618 2026-02-17T06:09:46.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 734439407618 2026-02-17T06:09:46.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407618' 2026-02-17T06:09:46.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:46.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:09:46.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=708669603845 2026-02-17T06:09:46.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 708669603845 2026-02-17T06:09:46.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407618 1-708669603845' 2026-02-17T06:09:46.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:46.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-734439407618 2026-02-17T06:09:46.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:46.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:09:46.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-734439407618 2026-02-17T06:09:46.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:46.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=734439407618 2026-02-17T06:09:46.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 734439407618' 2026-02-17T06:09:46.595 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 734439407618 2026-02-17T06:09:46.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:46.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 734439407618 2026-02-17T06:09:46.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:09:47.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:09:47.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:47.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407618 -lt 734439407618 2026-02-17T06:09:47.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:47.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-708669603845 2026-02-17T06:09:47.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:47.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:09:47.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-708669603845 2026-02-17T06:09:47.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:47.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=708669603845 2026-02-17T06:09:47.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 708669603845' 2026-02-17T06:09:47.903 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 708669603845 2026-02-17T06:09:47.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:09:48.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 708669603845 -lt 708669603845 2026-02-17T06:09:48.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:09:48.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:48.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:48.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:09:48.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:09:48.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:09:48.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:09:48.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:09:48.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:09:48.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:09:48.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:09:48.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:09:48.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:09:48.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:48.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:48.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:09:48.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:09:48.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:09:48.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1135: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:09:48.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:09:48.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:09:48.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:09:48.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:09:48.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:09:49.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T06:09:50.547 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T06:09:50.547 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T06:09:51.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:09:51.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:09:51.088 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:09:51.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:09:51.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:09:51.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:09:51.110 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:51.102+0000 7fbb6af1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:51.112 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:51.102+0000 7fbb6af1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:51.113 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:51.102+0000 7fbb6af1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:09:51.241 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:09:51.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:51.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:51.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:52.339 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:52.330+0000 7fbb6af1b8c0 -1 Falling back to public interface 2026-02-17T06:09:52.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:52.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:52.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:09:52.395 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:09:52.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:52.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:52.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:53.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:53.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:53.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:09:53.548 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:09:53.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:53.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:53.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:54.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:54.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:54.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:09:54.701 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:09:54.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:54.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:54.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:09:55.314 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:09:55.306+0000 7fbb6af1b8c0 -1 osd.1 172 log_to_monitors true 2026-02-17T06:09:55.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:09:55.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:09:55.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:09:55.855 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:09:55.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:09:55.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 176 up_thru 176 down_at 173 last_clean_interval [165,172) [v2:127.0.0.1:6810/43106631,v1:127.0.0.1:6811/43106631] [v2:127.0.0.1:6812/43106631,v1:127.0.0.1:6813/43106631] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:09:56.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:09:56.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:09:56.008 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:09:56.008 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:09:56.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:09:56.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:09:56.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:09:56.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:09:56.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:09:56.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:09:56.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:09:56.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:09:56.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:09:56.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:09:56.270 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:09:56.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:09:56.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:56.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:09:56.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=734439407621 2026-02-17T06:09:56.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 734439407621 2026-02-17T06:09:56.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407621' 2026-02-17T06:09:56.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:09:56.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:09:56.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=755914244098 2026-02-17T06:09:56.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 755914244098 2026-02-17T06:09:56.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407621 1-755914244098' 2026-02-17T06:09:56.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:56.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-734439407621 2026-02-17T06:09:56.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:56.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:09:56.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-734439407621 2026-02-17T06:09:56.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:56.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=734439407621 2026-02-17T06:09:56.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 734439407621' 2026-02-17T06:09:56.415 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 734439407621 2026-02-17T06:09:56.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:56.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407620 -lt 734439407621 2026-02-17T06:09:56.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:09:57.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:09:57.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:57.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407620 -lt 734439407621 2026-02-17T06:09:57.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:09:58.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:09:58.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:09:58.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407621 -lt 734439407621 2026-02-17T06:09:58.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:09:58.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-755914244098 2026-02-17T06:09:58.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:09:58.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:09:58.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-755914244098 2026-02-17T06:09:58.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:09:58.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=755914244098 2026-02-17T06:09:58.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 755914244098' 2026-02-17T06:09:58.890 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 755914244098 2026-02-17T06:09:58.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:09:59.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 755914244098 -lt 755914244098 2026-02-17T06:09:59.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:09:59.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:59.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:59.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:09:59.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:09:59.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:09:59.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:09:59.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:09:59.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:09:59.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:09:59.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:09:59.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:09:59.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:09:59.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:09:59.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:09:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:09:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:09:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:09:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1137: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-17T06:09:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:09:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:09:59.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:09:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:10:00.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:10:00.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-17T06:10:00.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:10:00.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:10:00.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:10:00.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:10:00.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:10:00.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 corrupt-info 2026-02-17T06:10:01.514 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T06:10:01.514 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T06:10:02.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:10:02.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:10:02.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:10:02.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:10:02.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:02.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:10:02.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:10:02.058 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:10:02.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:10:02.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:10:02.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:10:02.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:10:02.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:10:02.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:10:02.078 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:02.070+0000 7f946ef538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:02.080 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:02.070+0000 7f946ef538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:02.081 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:02.070+0000 7f946ef538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:02.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:10:02.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:02.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:03.327 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:03.318+0000 7f946ef538c0 -1 Falling back to public interface 2026-02-17T06:10:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:10:03.362 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:10:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:03.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:04.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:04.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:04.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:10:04.510 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:10:04.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:04.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:04.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:05.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:05.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:05.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:10:05.660 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:10:05.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:05.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:05.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:06.348 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:06.338+0000 7f946ef538c0 -1 osd.1 177 log_to_monitors true 2026-02-17T06:10:06.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:06.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:06.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:10:06.813 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:10:06.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:06.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 181 up_thru 181 down_at 178 last_clean_interval [176,177) [v2:127.0.0.1:6810/3031005443,v1:127.0.0.1:6811/3031005443] [v2:127.0.0.1:6812/3031005443,v1:127.0.0.1:6813/3031005443] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:10:06.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:10:06.964 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:10:06.964 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:10:06.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:10:06.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:10:06.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:10:07.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:10:07.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:10:07.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:10:07.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:10:07.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:10:07.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:10:07.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:10:07.231 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:10:07.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:10:07.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:07.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:10:07.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=734439407624 2026-02-17T06:10:07.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 734439407624 2026-02-17T06:10:07.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407624' 2026-02-17T06:10:07.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:07.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:10:07.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=777389080578 2026-02-17T06:10:07.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 777389080578 2026-02-17T06:10:07.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407624 1-777389080578' 2026-02-17T06:10:07.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:07.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-734439407624 2026-02-17T06:10:07.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:07.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:10:07.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-734439407624 2026-02-17T06:10:07.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:07.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=734439407624 2026-02-17T06:10:07.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 734439407624' 2026-02-17T06:10:07.378 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 734439407624 2026-02-17T06:10:07.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:07.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407623 -lt 734439407624 2026-02-17T06:10:07.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:08.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:10:08.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:08.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407624 -lt 734439407624 2026-02-17T06:10:08.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:08.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-777389080578 2026-02-17T06:10:08.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:08.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:10:08.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-777389080578 2026-02-17T06:10:08.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:08.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=777389080578 2026-02-17T06:10:08.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 777389080578' 2026-02-17T06:10:08.690 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 777389080578 2026-02-17T06:10:08.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:10:08.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 777389080578 -lt 777389080578 2026-02-17T06:10:08.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:10:08.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:08.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:09.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:10:09.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:10:09.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:10:09.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:10:09.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:10:09.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:10:09.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:10:09.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:10:09.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:10:09.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:10:09.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:09.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:09.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:10:09.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:10:09.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:10:09.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T06:10:09.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ19 2026-02-17T06:10:09.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 19 % 2 2026-02-17T06:10:09.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T06:10:09.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T06:10:09.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1146: TEST_corrupt_scrub_replicated: get_pg csr_pool ROBJ0 2026-02-17T06:10:09.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=csr_pool 2026-02-17T06:10:09.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=ROBJ0 2026-02-17T06:10:09.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-17T06:10:09.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:10:09.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1146: TEST_corrupt_scrub_replicated: local pg=3.0 2026-02-17T06:10:09.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1148: TEST_corrupt_scrub_replicated: ceph tell 'osd.*' injectargs -- --osd-max-object-size=1048576 2026-02-17T06:10:09.585 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: {} 2026-02-17T06:10:09.585 INFO:tasks.workunit.client.0.trial074.stderr:osd.0: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_max_object_size = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2026-02-17T06:10:09.593 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: {} 2026-02-17T06:10:09.593 INFO:tasks.workunit.client.0.trial074.stderr:osd.1: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_max_object_size = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2026-02-17T06:10:09.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1150: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ11 td/osd-scrub-repair 0 2026-02-17T06:10:09.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:10:09.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ11 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ11 2026-02-17T06:10:09.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:10:09.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ11 2026-02-17T06:10:09.603 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ11 2026-02-17T06:10:09.603 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:10:09.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:10:09.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:10:09.757 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T06:10:09.758 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:10:09.758 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:10:09.758 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:09.759 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:09.759 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:09.759 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:10:09.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:10:09.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:10:09.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:10:09.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:10:09.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T06:10:09.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:10:09.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:10:09.819 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:09.820 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:09.820 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:09.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:10:09.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:10:09.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.1.asok injectdataerr csr_pool ROBJ11 2026-02-17T06:10:09.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1151: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ12 td/osd-scrub-repair 1 2026-02-17T06:10:09.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:10:09.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:10:09.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T06:10:09.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ12 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ12 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:10:09.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ12 2026-02-17T06:10:09.881 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ12 2026-02-17T06:10:09.881 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:10:10.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:10:10.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T06:10:10.036 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:10:10.037 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:10:10.037 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:10.038 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:10.038 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:10.038 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:10:10.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:10:10.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:10:10.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:10:10.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:10:10.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T06:10:10.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:10:10.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:10:10.100 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:10.100 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:10.100 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:10.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:10:10.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:10:10.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.0.asok injectmdataerr csr_pool ROBJ12 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1152: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ13 td/osd-scrub-repair 1 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:10:10.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T06:10:10.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:10:10.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T06:10:10.160 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:10:10.160 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T06:10:10.160 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T06:10:10.160 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:10:10.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:10:10.309 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:10:10.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:10:10.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:10:10.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:10:10.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T06:10:10.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:10:10.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:10:10.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T06:10:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:10:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T06:10:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:10:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T06:10:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:10:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:10:10.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:10:10.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:10:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:10:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:10:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:10:10.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T06:10:10.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:10:10.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:10:10.376 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:10.376 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:10.376 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:10.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:10:10.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:10:10.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.0.asok injectmdataerr csr_pool ROBJ13 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1153: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ13 td/osd-scrub-repair 0 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T06:10:10.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:10:10.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T06:10:10.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:10:10.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T06:10:10.436 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T06:10:10.436 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:10:10.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:10:10.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T06:10:10.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:10:10.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T06:10:10.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:10:10.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T06:10:10.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:10:10.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:10:10.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:10:10.588 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T06:10:10.588 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:10:10.588 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:10:10.588 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:10.588 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:10.588 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:10.588 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:10:10.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:10:10.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:10:10.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:10:10.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:10:10.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T06:10:10.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:10:10.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:10:10.651 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:10.651 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:10.651 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:10.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:10:10.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:10:10.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.1.asok injectdataerr csr_pool ROBJ13 2026-02-17T06:10:10.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1155: TEST_corrupt_scrub_replicated: pg_scrub 3.0 2026-02-17T06:10:10.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T06:10:10.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T06:10:10.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:10:10.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:10:10.713 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:10:10.713 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:10:10.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:10:10.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:10:10.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:10:11.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:10:11.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:10:11.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:10:11.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:10:11.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:10:11.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:10:11.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:10:11.168 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:10:11.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:10:11.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:11.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:10:11.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=734439407626 2026-02-17T06:10:11.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 734439407626 2026-02-17T06:10:11.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407626' 2026-02-17T06:10:11.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:11.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:10:11.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=777389080579 2026-02-17T06:10:11.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 777389080579 2026-02-17T06:10:11.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-734439407626 1-777389080579' 2026-02-17T06:10:11.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:11.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-734439407626 2026-02-17T06:10:11.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:11.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:10:11.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-734439407626 2026-02-17T06:10:11.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:11.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=734439407626 2026-02-17T06:10:11.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 734439407626' 2026-02-17T06:10:11.316 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 734439407626 2026-02-17T06:10:11.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:11.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407624 -lt 734439407626 2026-02-17T06:10:11.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:12.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:10:12.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:12.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 734439407626 -lt 734439407626 2026-02-17T06:10:12.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:12.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-777389080579 2026-02-17T06:10:12.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:12.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:10:12.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-777389080579 2026-02-17T06:10:12.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=777389080579 2026-02-17T06:10:12.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 777389080579' 2026-02-17T06:10:12.635 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 777389080579 2026-02-17T06:10:12.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:10:12.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 777389080580 -lt 777389080579 2026-02-17T06:10:12.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:10:12.794 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:10:12.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:10:12.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:10:12.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:10:12.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:10:12.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:10:12.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:10:12.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:10:12.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:10:12.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:10:12.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:10:12.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T06:10:12.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:10:12.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:10:12.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:10:12.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:10:13.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:13.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T06:10:13.157 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-17T06:10:13.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:13.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:10:13.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:13.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:10:13.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:10:13.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:10:13.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:10:13.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:10:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:10:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:10:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:10:13.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:13.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:10:14.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:10:14.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:10:14.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:10:14.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:10:14.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:10:14.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:10:14.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:10:14.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:14.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:10:15.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:10:15.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:10:15.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:10:15.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:10:15.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:10:15.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:10:15.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:10:15.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:15.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:10:16.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:10:16.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:10:16.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:10:16.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:10:16.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:10:16.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:10:16.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:10:16.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:16.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:10:17.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:10:17.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:10:17.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:10:17.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:10:17.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:10:17.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:10:17.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:10:13.459993+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1157: TEST_corrupt_scrub_replicated: ERRORS=0 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1158: TEST_corrupt_scrub_replicated: declare -a s_err_strings 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1159: TEST_corrupt_scrub_replicated: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1160: TEST_corrupt_scrub_replicated: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : object info inconsistent ' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1161: TEST_corrupt_scrub_replicated: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1162: TEST_corrupt_scrub_replicated: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1163: TEST_corrupt_scrub_replicated: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1164: TEST_corrupt_scrub_replicated: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1165: TEST_corrupt_scrub_replicated: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1166: TEST_corrupt_scrub_replicated: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1167: TEST_corrupt_scrub_replicated: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1168: TEST_corrupt_scrub_replicated: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1169: TEST_corrupt_scrub_replicated: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1170: TEST_corrupt_scrub_replicated: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1171: TEST_corrupt_scrub_replicated: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1172: TEST_corrupt_scrub_replicated: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1173: TEST_corrupt_scrub_replicated: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : candidate size 1 info size 7 mismatch' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1174: TEST_corrupt_scrub_replicated: err_strings[15]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1175: TEST_corrupt_scrub_replicated: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1176: TEST_corrupt_scrub_replicated: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1177: TEST_corrupt_scrub_replicated: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049713/1049720 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1178: TEST_corrupt_scrub_replicated: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 1 missing, 8 inconsistent objects' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1179: TEST_corrupt_scrub_replicated: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 18 errors' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1180: TEST_corrupt_scrub_replicated: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:123a5f55:::ROBJ19:head : size 1049600 > 1048576 is too large' 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:17.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:17.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : candidate size 1 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049713/1049720 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 1 missing, 8 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 18 errors' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:10:18.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:123a5f55:::ROBJ19:head : size 1049600 > 1048576 is too large' td/osd-scrub-repair/osd.1.log 2026-02-17T06:10:18.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1191: TEST_corrupt_scrub_replicated: rados list-inconsistent-pg csr_pool 2026-02-17T06:10:18.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1193: TEST_corrupt_scrub_replicated: jq '. | length' td/osd-scrub-repair/json 2026-02-17T06:10:18.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1193: TEST_corrupt_scrub_replicated: test 1 = 1 2026-02-17T06:10:18.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1195: TEST_corrupt_scrub_replicated: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T06:10:18.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1195: TEST_corrupt_scrub_replicated: test 3.0 = 3.0 2026-02-17T06:10:18.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1197: TEST_corrupt_scrub_replicated: rados list-inconsistent-obj 3.0 2026-02-17T06:10:18.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1199: TEST_corrupt_scrub_replicated: jq .epoch td/osd-scrub-repair/json 2026-02-17T06:10:18.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1199: TEST_corrupt_scrub_replicated: epoch=181 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1201: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:10:18.172 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:10:18.173 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:10:18.173 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:10:18.173 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:10:18.173 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:10:18.173 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T06:10:18.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1201: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T06:10:18.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1201: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2082: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:10:18.191 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T06:10:18.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2082: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T06:10:18.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2082: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:10:18.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2083: TEST_corrupt_scrub_replicated: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:10:18.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:10:18.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2084: TEST_corrupt_scrub_replicated: test no = yes 2026-02-17T06:10:18.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2089: TEST_corrupt_scrub_replicated: test '' = yes 2026-02-17T06:10:18.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2094: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-17T06:10:18.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2096: TEST_corrupt_scrub_replicated: echo -n ZZZ 2026-02-17T06:10:18.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2097: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/change 2026-02-17T06:10:18.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2099: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:10:18.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:10:18.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:10:18.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:10:18.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:10:18.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:10:19.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T06:10:20.191 INFO:tasks.workunit.client.0.trial074.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T06:10:20.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:10:20.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:10:20.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:10:20.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:10:20.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:10:20.733 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:10:20.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:10:20.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:10:20.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:10:20.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:10:20.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:10:20.754 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:20.746+0000 7f4d7bc168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:20.755 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:20.746+0000 7f4d7bc168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:20.756 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:20.746+0000 7f4d7bc168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:20.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:10:20.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:10:20.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:10:20.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:10:20.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:10:20.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:20.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:10:20.882 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:10:20.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:20.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:21.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:22.003 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:21.994+0000 7f4d7bc168c0 -1 Falling back to public interface 2026-02-17T06:10:22.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:22.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:22.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:10:22.036 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:10:22.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:22.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:22.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:23.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:23.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:23.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:10:23.191 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:10:23.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:23.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:23.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:24.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:24.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:24.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:10:24.347 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:10:24.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:24.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:24.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:24.965 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:24.958+0000 7f4d7bc168c0 -1 osd.0 183 log_to_monitors true 2026-02-17T06:10:25.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:25.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:25.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:10:25.503 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:10:25.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:25.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:25.658 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 187 up_thru 187 down_at 184 last_clean_interval [171,183) [v2:127.0.0.1:6802/3945055430,v1:127.0.0.1:6803/3945055430] [v2:127.0.0.1:6804/3945055430,v1:127.0.0.1:6805/3945055430] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:10:25.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:10:25.660 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:10:25.660 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:10:25.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:10:25.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:10:25.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:10:25.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:10:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:10:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:10:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:10:25.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:10:25.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:10:25.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:10:25.925 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:10:25.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:10:25.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:25.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:10:25.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=803158884354 2026-02-17T06:10:25.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 803158884354 2026-02-17T06:10:25.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884354' 2026-02-17T06:10:25.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:25.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:10:26.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=777389080583 2026-02-17T06:10:26.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 777389080583 2026-02-17T06:10:26.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884354 1-777389080583' 2026-02-17T06:10:26.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:26.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-803158884354 2026-02-17T06:10:26.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:26.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:10:26.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-803158884354 2026-02-17T06:10:26.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:26.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=803158884354 2026-02-17T06:10:26.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 803158884354' 2026-02-17T06:10:26.072 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 803158884354 2026-02-17T06:10:26.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:26.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 803158884354 2026-02-17T06:10:26.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:27.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:10:27.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:27.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 803158884354 2026-02-17T06:10:27.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:28.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:10:28.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:28.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 803158884354 -lt 803158884354 2026-02-17T06:10:28.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:28.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-777389080583 2026-02-17T06:10:28.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:28.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:10:28.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-777389080583 2026-02-17T06:10:28.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:28.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=777389080583 2026-02-17T06:10:28.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 777389080583' 2026-02-17T06:10:28.545 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 777389080583 2026-02-17T06:10:28.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:10:28.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 777389080584 -lt 777389080583 2026-02-17T06:10:28.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:10:28.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:28.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:28.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:10:28.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:10:28.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:10:28.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:10:28.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:10:28.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:10:28.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:10:28.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:10:29.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:10:29.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:10:29.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:29.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:29.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:10:29.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:10:29.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:10:29.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2100: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi td/osd-scrub-repair/change 2026-02-17T06:10:29.219 INFO:tasks.workunit.client.0.trial074.stderr:rm: cannot remove 'td/osd-scrub-repair/oi': No such file or directory 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2102: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2103: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:10:29.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:10:29.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:10:29.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:10:29.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:10:29.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:10:29.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:10:30.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:10:30.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-17T06:10:30.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:10:30.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:10:30.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:10:30.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:10:30.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:10:30.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ10 get-attr _ 2026-02-17T06:10:30.843 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T06:10:30.843 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:10:31.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:10:31.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:10:31.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:10:31.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:10:31.379 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:10:31.379 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:31.379 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:31.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:10:31.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:10:31.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:10:31.382 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:10:31.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:10:31.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:10:31.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:10:31.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:10:31.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:10:31.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:10:31.402 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:31.394+0000 7f6b98ab48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:31.403 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:31.394+0000 7f6b98ab48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:31.404 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:31.394+0000 7f6b98ab48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:31.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:31.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:32.623 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:32.614+0000 7f6b98ab48c0 -1 Falling back to public interface 2026-02-17T06:10:32.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:32.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:32.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:10:32.690 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:10:32.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:32.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:32.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:33.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:33.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:33.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:10:33.842 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:10:33.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:33.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:33.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:34.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:34.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:34.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:10:34.993 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:10:34.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:34.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:35.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:35.628 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:35.618+0000 7f6b98ab48c0 -1 osd.1 188 log_to_monitors true 2026-02-17T06:10:36.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:36.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:36.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:10:36.145 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:10:36.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:36.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:36.297 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 192 up_thru 192 down_at 189 last_clean_interval [181,188) [v2:127.0.0.1:6810/1756908412,v1:127.0.0.1:6811/1756908412] [v2:127.0.0.1:6812/1756908412,v1:127.0.0.1:6813/1756908412] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:10:36.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:10:36.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:10:36.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:10:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:10:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:10:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:10:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:10:36.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:10:36.298 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:10:36.298 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:10:36.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:10:36.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:10:36.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:10:36.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:10:36.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:10:36.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:10:36.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:10:36.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:10:36.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:10:36.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:10:36.569 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:10:36.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:10:36.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:36.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:10:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=803158884357 2026-02-17T06:10:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 803158884357 2026-02-17T06:10:36.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884357' 2026-02-17T06:10:36.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:36.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:10:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=824633720834 2026-02-17T06:10:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 824633720834 2026-02-17T06:10:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884357 1-824633720834' 2026-02-17T06:10:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:36.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-803158884357 2026-02-17T06:10:36.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:10:36.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-803158884357 2026-02-17T06:10:36.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:36.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=803158884357 2026-02-17T06:10:36.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 803158884357' 2026-02-17T06:10:36.716 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 803158884357 2026-02-17T06:10:36.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:36.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 803158884356 -lt 803158884357 2026-02-17T06:10:36.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:37.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:10:37.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:38.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 803158884357 -lt 803158884357 2026-02-17T06:10:38.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:38.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-824633720834 2026-02-17T06:10:38.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:38.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:10:38.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-824633720834 2026-02-17T06:10:38.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:38.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=824633720834 2026-02-17T06:10:38.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 824633720834' 2026-02-17T06:10:38.034 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 824633720834 2026-02-17T06:10:38.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:10:38.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 824633720834 -lt 824633720834 2026-02-17T06:10:38.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:10:38.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:38.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:38.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:10:38.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:10:38.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:10:38.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:10:38.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:10:38.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:10:38.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:10:38.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:10:38.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:10:38.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:10:38.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:38.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:38.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:10:38.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:10:38.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:10:38.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2104: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ10 key2-ROBJ10 val2-ROBJ10 2026-02-17T06:10:38.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2105: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:38.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:10:38.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:10:38.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:10:38.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:10:38.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:10:38.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:10:40.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:40.676 INFO:tasks.workunit.client.0.trial074.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:10:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:10:41.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:10:41.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:10:41.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:10:41.219 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:10:41.219 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:41.219 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:10:41.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:10:41.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:10:41.221 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:10:41.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:10:41.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:10:41.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:10:41.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:10:41.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:10:41.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:10:41.240 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:41.230+0000 7fdcc9ad48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:41.242 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:41.234+0000 7fdcc9ad48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:41.243 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:41.234+0000 7fdcc9ad48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:10:41.375 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:10:41.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:41.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:41.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:42.487 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:42.478+0000 7fdcc9ad48c0 -1 Falling back to public interface 2026-02-17T06:10:42.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:42.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:42.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:10:42.526 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:10:42.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:42.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:42.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:43.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:43.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:43.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:10:43.681 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:10:43.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:43.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:43.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:44.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:44.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:44.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:10:44.831 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:10:44.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:44.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:44.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:45.503 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:45.494+0000 7fdcc9ad48c0 -1 osd.0 193 log_to_monitors true 2026-02-17T06:10:45.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:45.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:45.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:10:45.982 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:10:45.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:45.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 197 up_thru 197 down_at 194 last_clean_interval [187,193) [v2:127.0.0.1:6802/689188327,v1:127.0.0.1:6803/689188327] [v2:127.0.0.1:6804/689188327,v1:127.0.0.1:6805/689188327] exists,up 18ed35de-096b-4dff-b9f5-ec17fd7a8c19 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:10:46.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:10:46.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:10:46.130 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:10:46.131 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:10:46.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:10:46.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:10:46.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:10:46.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:10:46.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:10:46.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:10:46.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:10:46.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:10:46.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:10:46.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:10:46.397 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:10:46.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:10:46.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:46.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:10:46.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557314 2026-02-17T06:10:46.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557314 2026-02-17T06:10:46.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557314' 2026-02-17T06:10:46.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:46.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:10:46.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=824633720837 2026-02-17T06:10:46.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 824633720837 2026-02-17T06:10:46.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557314 1-824633720837' 2026-02-17T06:10:46.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:46.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557314 2026-02-17T06:10:46.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:46.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:10:46.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557314 2026-02-17T06:10:46.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:46.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557314 2026-02-17T06:10:46.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557314' 2026-02-17T06:10:46.543 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 846108557314 2026-02-17T06:10:46.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:46.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 846108557314 2026-02-17T06:10:46.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:47.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:10:47.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:47.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557314 -lt 846108557314 2026-02-17T06:10:47.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:47.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-824633720837 2026-02-17T06:10:47.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:47.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:10:47.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-824633720837 2026-02-17T06:10:47.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:47.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=824633720837 2026-02-17T06:10:47.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 824633720837' 2026-02-17T06:10:47.863 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 824633720837 2026-02-17T06:10:47.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:10:48.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 824633720837 -lt 824633720837 2026-02-17T06:10:48.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:10:48.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:48.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:48.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:10:48.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:10:48.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:10:48.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:10:48.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:10:48.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:10:48.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:10:48.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:10:48.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:10:48.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:10:48.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:48.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2106: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:10:48.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:10:48.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:10:48.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:10:48.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:10:48.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:10:48.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:10:49.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:10:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:10:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:10:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:10:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:10:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:10:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T06:10:50.479 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T06:10:50.479 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T06:10:51.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:10:51.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:10:51.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:10:51.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:10:51.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:10:51.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:10:51.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:10:51.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:10:51.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:10:51.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:10:51.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:51.027 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:10:51.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:10:51.030 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:10:51.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:10:51.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:10:51.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:10:51.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:10:51.050 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:51.042+0000 7f0014f2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:51.051 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:51.042+0000 7f0014f2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:51.052 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:51.042+0000 7f0014f2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:51.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:51.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:52.295 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:52.286+0000 7f0014f2e8c0 -1 Falling back to public interface 2026-02-17T06:10:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:10:52.346 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:10:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:52.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:52.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:53.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:53.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:53.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:10:53.498 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:10:53.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:53.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:53.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:54.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:54.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:54.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:10:54.649 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:10:54.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:54.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:10:55.323 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:10:55.314+0000 7f0014f2e8c0 -1 osd.1 198 log_to_monitors true 2026-02-17T06:10:55.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:10:55.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:10:55.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:10:55.800 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:10:55.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:10:55.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:10:55.953 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 202 up_thru 202 down_at 199 last_clean_interval [192,198) [v2:127.0.0.1:6810/1571464275,v1:127.0.0.1:6811/1571464275] [v2:127.0.0.1:6812/1571464275,v1:127.0.0.1:6813/1571464275] exists,up 85a1dc65-55a4-475c-a132-09f1a319858f 2026-02-17T06:10:55.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:10:55.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:10:55.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:10:55.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:10:55.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:10:55.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:10:55.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:10:55.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:10:55.955 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:10:55.955 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:10:55.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:10:55.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:10:55.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:10:56.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:10:56.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:10:56.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:10:56.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:10:56.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:10:56.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:10:56.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:10:56.227 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:10:56.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:10:56.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:56.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:10:56.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557317 2026-02-17T06:10:56.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557317 2026-02-17T06:10:56.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557317' 2026-02-17T06:10:56.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:10:56.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:10:56.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=867583393794 2026-02-17T06:10:56.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 867583393794 2026-02-17T06:10:56.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557317 1-867583393794' 2026-02-17T06:10:56.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:56.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557317 2026-02-17T06:10:56.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:56.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:10:56.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557317 2026-02-17T06:10:56.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:56.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557317 2026-02-17T06:10:56.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557317' 2026-02-17T06:10:56.376 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 846108557317 2026-02-17T06:10:56.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:56.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557316 -lt 846108557317 2026-02-17T06:10:56.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:57.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:10:57.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:57.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557316 -lt 846108557317 2026-02-17T06:10:57.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:10:58.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:10:58.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:10:58.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557317 -lt 846108557317 2026-02-17T06:10:58.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:10:58.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-867583393794 2026-02-17T06:10:58.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:10:58.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:10:58.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-867583393794 2026-02-17T06:10:58.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:10:58.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=867583393794 2026-02-17T06:10:58.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 867583393794' 2026-02-17T06:10:58.830 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 867583393794 2026-02-17T06:10:58.831 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:10:58.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 867583393794 -lt 867583393794 2026-02-17T06:10:58.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:10:58.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:58.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:59.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:10:59.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:10:59.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:10:59.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:10:59.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:10:59.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:10:59.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:10:59.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:10:59.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:10:59.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:10:59.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:10:59.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:10:59.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:10:59.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:10:59.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:10:59.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2107: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi 2026-02-17T06:10:59.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2109: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ11 td/osd-scrub-repair 0 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ11 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:10:59.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T06:10:59.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:10:59.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ11 2026-02-17T06:10:59.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:10:59.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ11 2026-02-17T06:10:59.502 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ11 2026-02-17T06:10:59.502 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:10:59.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:10:59.652 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:10:59.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:10:59.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T06:10:59.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:10:59.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:10:59.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:10:59.656 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T06:10:59.656 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:10:59.657 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:10:59.657 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:59.657 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:59.657 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:59.657 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:10:59.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:10:59.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:10:59.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:10:59.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:10:59.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T06:10:59.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:10:59.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:10:59.717 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:59.717 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:59.717 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:59.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:10:59.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:10:59.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.1.asok injectdataerr csr_pool ROBJ11 2026-02-17T06:10:59.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2110: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ12 td/osd-scrub-repair 1 2026-02-17T06:10:59.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:10:59.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:10:59.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ12 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ12 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:10:59.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ12 2026-02-17T06:10:59.778 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ12 2026-02-17T06:10:59.778 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:10:59.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:10:59.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T06:10:59.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:10:59.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T06:10:59.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:10:59.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T06:10:59.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:10:59.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:10:59.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:10:59.934 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T06:10:59.934 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:10:59.934 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:10:59.935 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:59.935 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:59.935 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:59.935 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:10:59.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:10:59.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:10:59.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:10:59.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:10:59.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T06:10:59.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:10:59.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:10:59.996 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:10:59.996 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:10:59.996 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:10:59.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:10:59.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:10:59.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.0.asok injectmdataerr csr_pool ROBJ12 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2111: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ13 td/osd-scrub-repair 1 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T06:11:00.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:11:00.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T06:11:00.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:11:00.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T06:11:00.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T06:11:00.056 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:11:00.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:11:00.209 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:11:00.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:11:00.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:11:00.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:11:00.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T06:11:00.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:11:00.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:11:00.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T06:11:00.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:11:00.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T06:11:00.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:11:00.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T06:11:00.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:11:00.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:11:00.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:11:00.212 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T06:11:00.212 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:11:00.212 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:11:00.213 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:11:00.213 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:00.213 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:00.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:11:00.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:11:00.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:11:00.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:11:00.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:11:00.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T06:11:00.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:11:00.273 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:11:00.274 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:11:00.274 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:00.274 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:00.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.0.asok 2026-02-17T06:11:00.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:11:00.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.0.asok injectmdataerr csr_pool ROBJ13 2026-02-17T06:11:00.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2112: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ13 td/osd-scrub-repair 0 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T06:11:00.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T06:11:00.334 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T06:11:00.334 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:11:00.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:11:00.485 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:11:00.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T06:11:00.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T06:11:00.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T06:11:00.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T06:11:00.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T06:11:00.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T06:11:00.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T06:11:00.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T06:11:00.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:11:00.488 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:00.489 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:00.489 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:11:00.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40489/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T06:11:00.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T06:11:00.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T06:11:00.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-osd.1.asok 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T06:11:00.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40489/ceph-osd.1.asok injectdataerr csr_pool ROBJ13 2026-02-17T06:11:00.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2115: TEST_corrupt_scrub_replicated: ceph tell 'osd.*' injectargs -- --osd-max-object-size=134217728 2026-02-17T06:11:00.673 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: {} 2026-02-17T06:11:00.673 INFO:tasks.workunit.client.0.trial074.stderr:osd.0: osd_max_object_size = '' 2026-02-17T06:11:00.681 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: {} 2026-02-17T06:11:00.681 INFO:tasks.workunit.client.0.trial074.stderr:osd.1: osd_max_object_size = '' 2026-02-17T06:11:00.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2117: TEST_corrupt_scrub_replicated: pg_deep_scrub 3.0 2026-02-17T06:11:00.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T06:11:00.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T06:11:00.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:11:00.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:11:00.689 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:11:00.689 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:11:00.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:11:00.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:11:00.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:11:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:11:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:11:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:11:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:11:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:11:00.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:11:01.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:11:01.133 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:11:01.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:11:01.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:01.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:11:01.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557319 2026-02-17T06:11:01.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557319 2026-02-17T06:11:01.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557319' 2026-02-17T06:11:01.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:01.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:11:01.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=867583393796 2026-02-17T06:11:01.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 867583393796 2026-02-17T06:11:01.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557319 1-867583393796' 2026-02-17T06:11:01.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:01.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557319 2026-02-17T06:11:01.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:01.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:11:01.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557319 2026-02-17T06:11:01.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:01.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557319 2026-02-17T06:11:01.280 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 846108557319 2026-02-17T06:11:01.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557319' 2026-02-17T06:11:01.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:01.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557317 -lt 846108557319 2026-02-17T06:11:01.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:11:02.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:11:02.441 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:02.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557319 -lt 846108557319 2026-02-17T06:11:02.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:02.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-867583393796 2026-02-17T06:11:02.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:02.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:11:02.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-867583393796 2026-02-17T06:11:02.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:02.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=867583393796 2026-02-17T06:11:02.596 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 867583393796 2026-02-17T06:11:02.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 867583393796' 2026-02-17T06:11:02.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:11:02.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 867583393796 -lt 867583393796 2026-02-17T06:11:02.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:11:02.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:11:02.768 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:11:02.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:11:02.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:11:02.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:11:02.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:11:02.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:11:02.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:11:02.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:11:02.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:11:02.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:11:02.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:11:02.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:02.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:11:02.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:02.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:11:02.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T06:06:22.644385+0000 2026-02-17T06:11:02.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T06:11:03.127 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T06:06:22.644385+0000 last_deep_scrub_stamp 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:06:22.644385+0000 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:03.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:11:03.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:11:03.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:11:04.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:11:04.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:04.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:11:04.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:04.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:11:04.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:04.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:11:04.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:11:04.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:11:05.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:11:05.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:05.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:11:05.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:05.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:11:05.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:05.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:11:05.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:06:22.644385+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:11:05.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:11:06.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:11:06.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:06.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T06:11:06.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:06.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:11:06.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:06.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T06:11:06.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:03.218694+0000 '>' 2026-02-17T06:06:22.644385+0000 2026-02-17T06:11:06.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:11:06.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2119: TEST_corrupt_scrub_replicated: err_strings=() 2026-02-17T06:11:06.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2120: TEST_corrupt_scrub_replicated: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' 2026-02-17T06:11:06.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2121: TEST_corrupt_scrub_replicated: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)], object info inconsistent ' 2026-02-17T06:11:06.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2122: TEST_corrupt_scrub_replicated: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)]' 2026-02-17T06:11:06.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2123: TEST_corrupt_scrub_replicated: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : failed to pick suitable auth object' 2026-02-17T06:11:06.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2124: TEST_corrupt_scrub_replicated: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2125: TEST_corrupt_scrub_replicated: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2126: TEST_corrupt_scrub_replicated: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2127: TEST_corrupt_scrub_replicated: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2128: TEST_corrupt_scrub_replicated: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:87abbf36:::ROBJ11:head : candidate had a read error' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2129: TEST_corrupt_scrub_replicated: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2130: TEST_corrupt_scrub_replicated: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2131: TEST_corrupt_scrub_replicated: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8aa5320e:::ROBJ17:head : failed to pick suitable auth object' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2132: TEST_corrupt_scrub_replicated: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0xefced57a != omap_digest 0x6a73cc07 from shard 1' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2133: TEST_corrupt_scrub_replicated: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0x6a73cc07 != omap_digest 0xefced57a from auth oi 3:8b55fa4b:::ROBJ7:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 21 dd 2ddbf8f5 od efced57a alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2134: TEST_corrupt_scrub_replicated: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:a53c12e8:::ROBJ6:head : omap_digest 0x689ee887 != omap_digest 0x179c919f from shard 1, omap_digest 0x689ee887 != omap_digest 0x179c919f from auth oi 3:a53c12e8:::ROBJ6:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 18 dd 2ddbf8f5 od 179c919f alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2135: TEST_corrupt_scrub_replicated: err_strings[15]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2136: TEST_corrupt_scrub_replicated: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2137: TEST_corrupt_scrub_replicated: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:b1f19cbd:::ROBJ10:head : failed to pick suitable auth object' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2138: TEST_corrupt_scrub_replicated: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2139: TEST_corrupt_scrub_replicated: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2140: TEST_corrupt_scrub_replicated: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2141: TEST_corrupt_scrub_replicated: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2142: TEST_corrupt_scrub_replicated: err_strings[22]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2143: TEST_corrupt_scrub_replicated: err_strings[23]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from shard 0, data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' 2026-02-17T06:11:06.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2144: TEST_corrupt_scrub_replicated: err_strings[24]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:d60617f9:::ROBJ13:head : candidate had a read error' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2145: TEST_corrupt_scrub_replicated: err_strings[25]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2146: TEST_corrupt_scrub_replicated: err_strings[26]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:d60617f9:::ROBJ13:head : failed to pick suitable object info' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2147: TEST_corrupt_scrub_replicated: err_strings[27]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:e97ce31e:::ROBJ2:head : data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from shard 1, data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from auth oi 3:e97ce31e:::ROBJ2:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 6 dd 2ddbf8f5 od f8e11918 alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2148: TEST_corrupt_scrub_replicated: err_strings[28]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2149: TEST_corrupt_scrub_replicated: err_strings[29]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:f4981d31:::ROBJ4:head : omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from shard 1, omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from auth oi 3:f4981d31:::ROBJ4:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 12 dd 2ddbf8f5 od e2d46ea4 alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2150: TEST_corrupt_scrub_replicated: err_strings[30]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x1a862a41 != omap_digest 0x6cac8f6 from shard 1' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2151: TEST_corrupt_scrub_replicated: err_strings[31]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x6cac8f6 != omap_digest 0x1a862a41 from auth oi 3:f4bfd4d1:::ROBJ5:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 15 dd 2ddbf8f5 od 1a862a41 alloc_hint [[]0 0 0[]][)]' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2152: TEST_corrupt_scrub_replicated: err_strings[32]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : candidate size 3 info size 7 mismatch' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2153: TEST_corrupt_scrub_replicated: err_strings[33]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' 2026-02-17T06:11:06.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2154: TEST_corrupt_scrub_replicated: err_strings[34]='log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' 2026-02-17T06:11:06.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2155: TEST_corrupt_scrub_replicated: err_strings[35]='log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' 2026-02-17T06:11:06.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2156: TEST_corrupt_scrub_replicated: err_strings[36]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049715/1049716 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-17T06:11:06.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2157: TEST_corrupt_scrub_replicated: err_strings[37]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 1 missing, 11 inconsistent objects' 2026-02-17T06:11:06.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2158: TEST_corrupt_scrub_replicated: err_strings[38]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 35 errors' 2026-02-17T06:11:06.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)], object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:87abbf36:::ROBJ11:head : candidate had a read error' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8aa5320e:::ROBJ17:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0xefced57a != omap_digest 0x6a73cc07 from shard 1' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0x6a73cc07 != omap_digest 0xefced57a from auth oi 3:8b55fa4b:::ROBJ7:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 21 dd 2ddbf8f5 od efced57a alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:a53c12e8:::ROBJ6:head : omap_digest 0x689ee887 != omap_digest 0x179c919f from shard 1, omap_digest 0x689ee887 != omap_digest 0x179c919f from auth oi 3:a53c12e8:::ROBJ6:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 18 dd 2ddbf8f5 od 179c919f alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:b1f19cbd:::ROBJ10:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from shard 0, data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:d60617f9:::ROBJ13:head : candidate had a read error' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:d60617f9:::ROBJ13:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:e97ce31e:::ROBJ2:head : data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from shard 1, data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from auth oi 3:e97ce31e:::ROBJ2:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 6 dd 2ddbf8f5 od f8e11918 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:f4981d31:::ROBJ4:head : omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from shard 1, omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from auth oi 3:f4981d31:::ROBJ4:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 12 dd 2ddbf8f5 od e2d46ea4 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:06.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:06.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x1a862a41 != omap_digest 0x6cac8f6 from shard 1' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x6cac8f6 != omap_digest 0x1a862a41 from auth oi 3:f4bfd4d1:::ROBJ5:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 15 dd 2ddbf8f5 od 1a862a41 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : candidate size 3 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049715/1049716 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 1 missing, 11 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T06:11:07.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 35 errors' td/osd-scrub-repair/osd.1.log 2026-02-17T06:11:07.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2169: TEST_corrupt_scrub_replicated: rados list-inconsistent-pg csr_pool 2026-02-17T06:11:07.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2171: TEST_corrupt_scrub_replicated: jq '. | length' td/osd-scrub-repair/json 2026-02-17T06:11:07.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2171: TEST_corrupt_scrub_replicated: test 1 = 1 2026-02-17T06:11:07.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2173: TEST_corrupt_scrub_replicated: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T06:11:07.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2173: TEST_corrupt_scrub_replicated: test 3.0 = 3.0 2026-02-17T06:11:07.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2175: TEST_corrupt_scrub_replicated: rados list-inconsistent-obj 3.0 2026-02-17T06:11:07.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2177: TEST_corrupt_scrub_replicated: jq .epoch td/osd-scrub-repair/json 2026-02-17T06:11:07.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2177: TEST_corrupt_scrub_replicated: epoch=202 2026-02-17T06:11:07.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2179: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T06:11:07.157 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:11:07.157 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:11:07.157 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:11:07.157 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:11:07.157 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:11:07.157 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2179: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T06:11:07.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2179: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:11:07.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3558: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T06:11:07.181 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:11:07.181 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:11:07.181 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:11:07.181 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:11:07.181 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3558: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T06:11:07.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3558: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:11:07.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3559: TEST_corrupt_scrub_replicated: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:11:07.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:11:07.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3560: TEST_corrupt_scrub_replicated: test no = yes 2026-02-17T06:11:07.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3565: TEST_corrupt_scrub_replicated: test '' = yes 2026-02-17T06:11:07.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3570: TEST_corrupt_scrub_replicated: repair 3.0 2026-02-17T06:11:07.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=3.0 2026-02-17T06:11:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 3.0 2026-02-17T06:11:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:11:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:07.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:11:07.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:07.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 3.0 2026-02-17T06:11:07.508 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0 on osd.1 to repair 2026-02-17T06:11:07.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 3.0 2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:07.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:11:07.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:07.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:11:07.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:11:07.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:07.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:11:07.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:07.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:11:07.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:07.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:11:07.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:03.218694+0000 '>' 2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:07.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:11:08.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:11:08.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:08.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:11:08.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:08.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:11:08.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:08.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:11:08.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:03.218694+0000 '>' 2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:08.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:11:09.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:11:09.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:09.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:11:09.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:09.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:11:09.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:09.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:11:09.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:03.218694+0000 '>' 2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:09.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:11:10.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:11:10.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:10.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:11:10.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:10.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:11:10.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:10.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:11:11.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:03.218694+0000 '>' 2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:11.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:11:12.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:11:12.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:11:12.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:11:12.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:11:12.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:11:12.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:11:12.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:11:12.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:08.237116+0000 '>' 2026-02-17T06:11:03.218694+0000 2026-02-17T06:11:12.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:11:12.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3571: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-17T06:11:12.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:11:12.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:11:12.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:11:12.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:11:12.281 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:11:12.281 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:11:12.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:11:12.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:11:12.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:11:12.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:11:12.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:11:12.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:11:12.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:11:12.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:11:12.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:11:12.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:11:12.546 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:11:12.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:11:12.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:12.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:11:12.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557322 2026-02-17T06:11:12.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557322 2026-02-17T06:11:12.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557322' 2026-02-17T06:11:12.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:12.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:11:12.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=867583393799 2026-02-17T06:11:12.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 867583393799 2026-02-17T06:11:12.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557322 1-867583393799' 2026-02-17T06:11:12.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:12.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557322 2026-02-17T06:11:12.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:12.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:11:12.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557322 2026-02-17T06:11:12.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:12.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557322 2026-02-17T06:11:12.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557322' 2026-02-17T06:11:12.696 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 846108557322 2026-02-17T06:11:12.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:12.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557321 -lt 846108557322 2026-02-17T06:11:12.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:11:13.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:11:13.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:14.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557322 -lt 846108557322 2026-02-17T06:11:14.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:14.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-867583393799 2026-02-17T06:11:14.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:14.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:11:14.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-867583393799 2026-02-17T06:11:14.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:14.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=867583393799 2026-02-17T06:11:14.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 867583393799' 2026-02-17T06:11:14.013 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 867583393799 2026-02-17T06:11:14.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:11:14.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 867583393799 -lt 867583393799 2026-02-17T06:11:14.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:11:14.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:14.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:11:14.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:11:14.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:11:14.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:11:14.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:11:14.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:11:14.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:11:14.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:11:14.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:11:14.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:11:14.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:14.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:14.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:11:14.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:11:14.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:11:14.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3574: TEST_corrupt_scrub_replicated: timeout 30 rados -p csr_pool get ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-17T06:11:14.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3575: TEST_corrupt_scrub_replicated: timeout 30 rados -p csr_pool get ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-17T06:11:14.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3577: TEST_corrupt_scrub_replicated: diff -q td/osd-scrub-repair/new.ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-17T06:11:14.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3578: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-17T06:11:14.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3579: TEST_corrupt_scrub_replicated: diff -q td/osd-scrub-repair/new.ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-17T06:11:14.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3580: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-17T06:11:14.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3582: TEST_corrupt_scrub_replicated: '[' 0 '!=' 0 ']' 2026-02-17T06:11:14.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3588: TEST_corrupt_scrub_replicated: ceph osd pool rm csr_pool csr_pool --yes-i-really-really-mean-it 2026-02-17T06:11:14.922 INFO:tasks.workunit.client.0.trial074.stderr:pool 'csr_pool' removed 2026-02-17T06:11:14.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:11:14.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:11:15.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:11:15.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:11:15.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:11:15.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:11:15.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:11:15.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:11:15.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:11:15.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:11:15.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:11:15.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:11:15.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:11:15.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:11:15.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:11:15.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:11:15.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:11:15.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:11:15.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:11:15.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:11:15.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:11:15.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:11:15.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:11:15.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:11:15.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:11:15.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:11:15.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:11:15.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:11:15.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:11:15.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:11:15.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:11:15.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:11:15.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:11:15.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:11:15.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:11:15.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:11:15.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:11:15.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:11:15.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:11:15.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:11:15.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:11:15.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:11:15.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:11:15.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:11:15.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:11:15.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:11:15.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:11:15.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:11:15.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:11:15.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:11:15.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:11:15.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:11:15.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:11:15.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:11:15.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:11:15.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:11:15.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_snapset_scrub_rep td/osd-scrub-repair 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5931: TEST_corrupt_snapset_scrub_rep: local dir=td/osd-scrub-repair 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5932: TEST_corrupt_snapset_scrub_rep: local poolname=csr_pool 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5933: TEST_corrupt_snapset_scrub_rep: local total_objs=2 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5935: TEST_corrupt_snapset_scrub_rep: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:11:15.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T06:11:15.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:11:15.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:11:15.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:11:15.133 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:11:15.133 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.134 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:15.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T06:11:15.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:11:15.166 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:11:15.167 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:11:15.167 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.167 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.167 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:11:15.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:11:15.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:11:15.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:11:15.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:11:15.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:11:15.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:11:15.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:11:15.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:11:15.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5936: TEST_corrupt_snapset_scrub_rep: run_mgr td/osd-scrub-repair x 2026-02-17T06:11:15.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:11:15.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:11:15.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:11:15.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:11:15.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:11:15.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:11:15.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:11:15.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:11:15.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:11:15.371 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:11:15.371 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.371 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:15.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:11:15.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:11:15.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5937: TEST_corrupt_snapset_scrub_rep: run_osd td/osd-scrub-repair 0 2026-02-17T06:11:15.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:11:15.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:11:15.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:11:15.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:11:15.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:11:15.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0f58482f-cd4e-42b5-a605-588ed4760dcd 2026-02-17T06:11:15.400 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 0f58482f-cd4e-42b5-a605-588ed4760dcd 2026-02-17T06:11:15.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 0f58482f-cd4e-42b5-a605-588ed4760dcd' 2026-02-17T06:11:15.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:11:15.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCDBpRp+i0hGBAAbZxw1MISZk6ES5PPwjFvgA== 2026-02-17T06:11:15.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCDBpRp+i0hGBAAbZxw1MISZk6ES5PPwjFvgA=="}' 2026-02-17T06:11:15.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0f58482f-cd4e-42b5-a605-588ed4760dcd -i td/osd-scrub-repair/0/new.json 2026-02-17T06:11:15.500 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:11:15.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:11:15.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCDBpRp+i0hGBAAbZxw1MISZk6ES5PPwjFvgA== --osd-uuid 0f58482f-cd4e-42b5-a605-588ed4760dcd 2026-02-17T06:11:15.525 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:15.514+0000 7fbd969b88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:15.526 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:15.518+0000 7fbd969b88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:15.527 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:15.518+0000 7fbd969b88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:15.527 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:15.518+0000 7fbd969b88c0 -1 bdev(0x5628f6e3fc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:11:15.528 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:15.518+0000 7fbd969b88c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:11:18.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:11:18.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:11:18.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:11:18.331 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:11:18.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:11:18.520 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:11:18.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:11:18.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:11:18.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:11:18.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:11:18.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:11:18.543 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:18.534+0000 7f17435b38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:18.544 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:18.534+0000 7f17435b38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:18.546 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:18.538+0000 7f17435b38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:18.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:11:18.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:11:18.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:11:18.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:11:18.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:11:18.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:18.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:11:18.675 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:11:18.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:18.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:18.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:19.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:19.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:19.831 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:11:19.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:11:19.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:19.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:19.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:20.243 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:20.234+0000 7f17435b38c0 -1 Falling back to public interface 2026-02-17T06:11:20.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:20.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:20.982 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:11:20.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:11:20.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:20.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:21.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:22.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:22.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:22.131 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:11:22.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:11:22.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:22.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:22.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:23.248 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:23.238+0000 7f17435b38c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:11:23.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:23.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:23.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:11:23.282 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:11:23.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:23.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:23.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:24.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:24.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:24.436 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:11:24.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:11:24.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:24.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:24.585 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/552963325,v1:127.0.0.1:6803/552963325] [v2:127.0.0.1:6804/552963325,v1:127.0.0.1:6805/552963325] exists,up 0f58482f-cd4e-42b5-a605-588ed4760dcd 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5938: TEST_corrupt_snapset_scrub_rep: run_osd td/osd-scrub-repair 1 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:11:24.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:11:24.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:11:24.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:11:24.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:11:24.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:11:24.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2b54aa39-2573-4522-851a-fe80ccd741d3 2026-02-17T06:11:24.593 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 2b54aa39-2573-4522-851a-fe80ccd741d3 2026-02-17T06:11:24.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 2b54aa39-2573-4522-851a-fe80ccd741d3' 2026-02-17T06:11:24.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:11:24.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCMBpRpWVizIxAAGeprNQryKImlimml0Yeivw== 2026-02-17T06:11:24.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCMBpRpWVizIxAAGeprNQryKImlimml0Yeivw=="}' 2026-02-17T06:11:24.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2b54aa39-2573-4522-851a-fe80ccd741d3 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:11:24.748 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:11:24.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:11:24.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCMBpRpWVizIxAAGeprNQryKImlimml0Yeivw== --osd-uuid 2b54aa39-2573-4522-851a-fe80ccd741d3 2026-02-17T06:11:24.774 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:24.766+0000 7fcab5b6b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:24.775 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:24.766+0000 7fcab5b6b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:24.776 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:24.766+0000 7fcab5b6b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:24.777 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:24.766+0000 7fcab5b6b8c0 -1 bdev(0x55f61d4b7c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:11:24.777 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:24.770+0000 7fcab5b6b8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:11:27.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:11:27.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:11:27.084 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:11:27.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:11:27.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:11:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:11:27.277 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:11:27.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:11:27.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:11:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:11:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:11:27.298 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:27.290+0000 7fde85c4f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:27.300 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:27.290+0000 7fde85c4f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:27.301 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:27.294+0000 7fde85c4f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:27.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:11:27.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:11:27.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:11:27.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:11:27.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:11:27.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:27.433 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:11:27.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:11:27.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:27.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:27.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:28.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:28.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:28.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:11:28.584 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:11:28.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:28.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:28.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:28.766 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:28.758+0000 7fde85c4f8c0 -1 Falling back to public interface 2026-02-17T06:11:29.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:29.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:29.736 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:11:29.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:11:29.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:29.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:29.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:30.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:30.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:30.886 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:11:30.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:11:30.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:30.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:31.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:31.740 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:31.730+0000 7fde85c4f8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:11:32.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:32.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:32.036 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:11:32.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:11:32.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:32.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:32.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:33.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:33.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:33.189 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:11:33.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:11:33.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:33.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:33.340 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3427567451,v1:127.0.0.1:6811/3427567451] [v2:127.0.0.1:6812/3427567451,v1:127.0.0.1:6813/3427567451] exists,up 2b54aa39-2573-4522-851a-fe80ccd741d3 2026-02-17T06:11:33.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:11:33.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:11:33.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:11:33.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5939: TEST_corrupt_snapset_scrub_rep: create_rbd_pool 2026-02-17T06:11:33.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:11:33.479 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:11:33.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:11:33.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:11:33.702 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:11:33.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:11:34.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:11:34.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5940: TEST_corrupt_snapset_scrub_rep: wait_for_clean 2026-02-17T06:11:34.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:11:34.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:11:34.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:11:34.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:11:34.996 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:11:34.996 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:11:34.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:11:34.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:11:34.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:11:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:11:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:11:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:11:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:11:35.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:11:35.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:11:35.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:11:35.250 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:11:35.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:11:35.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:35.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:11:35.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:11:35.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:11:35.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:11:35.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:35.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:11:35.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T06:11:35.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T06:11:35.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T06:11:35.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:35.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:11:35.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:35.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:11:35.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:11:35.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:35.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:11:35.400 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:11:35.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:11:35.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:35.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T06:11:35.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:11:36.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:11:36.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:36.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:11:36.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:36.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T06:11:36.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:36.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:11:36.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T06:11:36.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T06:11:36.713 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672962 2026-02-17T06:11:36.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T06:11:36.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:11:36.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T06:11:36.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:11:36.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:36.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:37.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:11:37.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:11:37.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:11:37.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:11:37.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:11:37.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:11:37.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:11:37.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:11:37.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:11:37.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:11:37.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:37.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:37.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:11:37.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:11:37.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:11:37.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5942: TEST_corrupt_snapset_scrub_rep: create_pool foo 1 2026-02-17T06:11:37.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T06:11:37.580 INFO:tasks.workunit.client.0.trial074.stderr:pool 'foo' created 2026-02-17T06:11:37.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:11:38.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5943: TEST_corrupt_snapset_scrub_rep: create_pool csr_pool 1 1 2026-02-17T06:11:38.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create csr_pool 1 1 2026-02-17T06:11:38.790 INFO:tasks.workunit.client.0.trial074.stderr:pool 'csr_pool' created 2026-02-17T06:11:38.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:11:39.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5944: TEST_corrupt_snapset_scrub_rep: wait_for_clean 2026-02-17T06:11:39.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:11:39.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:11:39.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:11:39.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:11:39.804 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:11:39.804 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:11:39.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:11:39.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:11:39.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:11:39.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:11:39.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:11:39.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:11:39.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:11:39.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:11:39.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:11:40.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:11:40.073 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:11:40.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:11:40.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:40.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:11:40.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T06:11:40.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T06:11:40.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T06:11:40.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:40.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:11:40.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T06:11:40.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T06:11:40.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964' 2026-02-17T06:11:40.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:40.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T06:11:40.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:40.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:11:40.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T06:11:40.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:40.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T06:11:40.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T06:11:40.220 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836486 2026-02-17T06:11:40.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:40.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T06:11:40.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:11:41.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:11:41.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:41.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T06:11:41.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:11:42.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:11:42.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:42.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T06:11:42.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:42.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T06:11:42.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:42.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:11:42.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T06:11:42.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:42.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T06:11:42.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T06:11:42.697 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T06:11:42.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:11:42.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T06:11:42.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:11:42.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:42.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:43.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:11:43.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:11:43.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:11:43.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:11:43.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:11:43.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:11:43.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:11:43.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:11:43.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:11:43.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:11:43.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:43.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:43.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:11:43.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:11:43.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:11:43.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5946: TEST_corrupt_snapset_scrub_rep: seq 1 2 2026-02-17T06:11:43.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5946: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T06:11:43.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5947: TEST_corrupt_snapset_scrub_rep: objname=ROBJ1 2026-02-17T06:11:43.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5948: TEST_corrupt_snapset_scrub_rep: add_something td/osd-scrub-repair csr_pool ROBJ1 2026-02-17T06:11:43.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:11:43.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:11:43.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ1 2026-02-17T06:11:43.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:11:43.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:11:43.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:11:43.562 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:11:43.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:11:43.766 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:11:43.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:11:43.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:11:43.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:11:43.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5950: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapheader ROBJ1 hdr-ROBJ1 2026-02-17T06:11:43.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5951: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapval ROBJ1 key-ROBJ1 val-ROBJ1 2026-02-17T06:11:43.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5946: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T06:11:43.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5947: TEST_corrupt_snapset_scrub_rep: objname=ROBJ2 2026-02-17T06:11:43.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5948: TEST_corrupt_snapset_scrub_rep: add_something td/osd-scrub-repair csr_pool ROBJ2 2026-02-17T06:11:43.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:11:43.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T06:11:43.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ2 2026-02-17T06:11:43.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:11:43.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:11:43.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:11:44.058 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:11:44.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:11:44.262 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:11:44.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:11:44.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:11:44.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:11:44.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5950: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapheader ROBJ2 hdr-ROBJ2 2026-02-17T06:11:44.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5951: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapval ROBJ2 key-ROBJ2 val-ROBJ2 2026-02-17T06:11:44.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5954: TEST_corrupt_snapset_scrub_rep: get_pg csr_pool ROBJ0 2026-02-17T06:11:44.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=csr_pool 2026-02-17T06:11:44.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=ROBJ0 2026-02-17T06:11:44.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-17T06:11:44.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:11:44.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5954: TEST_corrupt_snapset_scrub_rep: local pg=3.0 2026-02-17T06:11:44.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5955: TEST_corrupt_snapset_scrub_rep: get_primary csr_pool ROBJ0 2026-02-17T06:11:44.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=csr_pool 2026-02-17T06:11:44.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=ROBJ0 2026-02-17T06:11:44.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map csr_pool ROBJ0 2026-02-17T06:11:44.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:11:44.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5955: TEST_corrupt_snapset_scrub_rep: local primary=1 2026-02-17T06:11:44.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5957: TEST_corrupt_snapset_scrub_rep: rados -p csr_pool mksnap snap1 2026-02-17T06:11:44.768 INFO:tasks.workunit.client.0.trial074.stdout:created pool csr_pool snap snap1 2026-02-17T06:11:44.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5958: TEST_corrupt_snapset_scrub_rep: echo -n head_of_snapshot_data 2026-02-17T06:11:44.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5960: TEST_corrupt_snapset_scrub_rep: seq 1 2 2026-02-17T06:11:44.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5960: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T06:11:44.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5961: TEST_corrupt_snapset_scrub_rep: objname=ROBJ1 2026-02-17T06:11:44.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: expr 1 % 2 2026-02-17T06:11:44.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: local osd=1 2026-02-17T06:11:44.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5966: TEST_corrupt_snapset_scrub_rep: case $i in 2026-02-17T06:11:44.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5968: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/change 2026-02-17T06:11:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5969: TEST_corrupt_snapset_scrub_rep: objectstore_tool td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-17T06:11:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:11:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:11:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:11:44.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:11:44.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:11:46.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:11:46.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-17T06:11:46.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:11:46.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:11:46.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:11:46.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:11:46.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:11:46.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 --head ROBJ1 clear-snapset corrupt 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:11:47.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:11:47.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:11:47.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:11:47.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:11:47.300 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:11:47.300 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:47.300 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:47.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:11:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:11:47.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:11:47.302 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:11:47.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:11:47.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:11:47.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:11:47.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:11:47.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:11:47.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:11:47.320 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:47.310+0000 7fb9c41148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:47.321 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:47.314+0000 7fb9c41148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:47.322 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:47.314+0000 7fb9c41148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:11:47.459 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:11:47.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:47.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:47.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:48.574 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:48.566+0000 7fb9c41148c0 -1 Falling back to public interface 2026-02-17T06:11:48.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:48.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:48.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:11:48.611 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:11:48.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:48.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:48.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:49.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:49.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:49.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:11:49.760 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:11:49.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:49.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:49.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:50.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:50.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:50.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:11:50.906 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:11:50.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:50.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:51.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:51.588 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:51.578+0000 7fb9c41148c0 -1 osd.1 29 log_to_monitors true 2026-02-17T06:11:52.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:52.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:52.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:11:52.055 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:11:52.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:52.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:11:52.202 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 33 up_thru 33 down_at 30 last_clean_interval [10,29) [v2:127.0.0.1:6810/3738463227,v1:127.0.0.1:6811/3738463227] [v2:127.0.0.1:6812/3738463227,v1:127.0.0.1:6813/3738463227] exists,up 2b54aa39-2573-4522-851a-fe80ccd741d3 2026-02-17T06:11:52.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:11:52.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:11:52.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:11:52.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:11:52.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:11:52.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:11:52.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:11:52.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:11:52.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:11:52.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:11:52.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:11:52.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:11:52.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:11:52.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:11:52.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:11:52.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:11:52.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:11:52.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:11:52.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:11:52.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:11:52.475 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:11:52.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:11:52.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:52.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:11:52.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T06:11:52.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T06:11:52.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T06:11:52.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:11:52.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:11:52.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920770 2026-02-17T06:11:52.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920770 2026-02-17T06:11:52.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-141733920770' 2026-02-17T06:11:52.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:52.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T06:11:52.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:52.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:11:52.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T06:11:52.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:52.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T06:11:52.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T06:11:52.624 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836489 2026-02-17T06:11:52.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:52.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T06:11:52.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:11:53.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:11:53.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:11:53.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T06:11:53.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:11:53.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920770 2026-02-17T06:11:53.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:11:53.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:11:53.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920770 2026-02-17T06:11:53.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:11:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920770 2026-02-17T06:11:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920770' 2026-02-17T06:11:53.944 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 141733920770 2026-02-17T06:11:53.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:11:54.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920770 2026-02-17T06:11:54.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:11:54.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:54.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:54.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:11:54.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:11:54.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:11:54.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:11:54.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:11:54.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:11:54.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:11:54.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:11:54.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:11:54.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:11:54.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:11:54.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:11:54.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:11:54.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:11:54.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:11:54.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5960: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T06:11:54.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5961: TEST_corrupt_snapset_scrub_rep: objname=ROBJ2 2026-02-17T06:11:54.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: expr 2 % 2 2026-02-17T06:11:54.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: local osd=0 2026-02-17T06:11:54.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5966: TEST_corrupt_snapset_scrub_rep: case $i in 2026-02-17T06:11:54.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5973: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/change 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5974: TEST_corrupt_snapset_scrub_rep: objectstore_tool td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:11:54.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:11:54.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:11:54.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:11:54.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:11:54.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:11:54.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:11:55.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:11:55.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-17T06:11:55.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:11:55.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:11:55.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:11:55.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:11:55.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:11:55.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 --head ROBJ2 clear-snapset corrupt 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:11:57.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:11:57.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:11:57.135 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:11:57.135 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:11:57.135 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:11:57.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:11:57.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:11:57.137 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:11:57.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:11:57.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:11:57.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:11:57.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:11:57.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:11:57.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:11:57.150 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:57.142+0000 7f9a67a818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:57.151 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:57.142+0000 7f9a67a818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:57.152 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:57.142+0000 7f9a67a818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:57.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:57.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:58.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:11:58.406+0000 7f9a67a818c0 -1 Falling back to public interface 2026-02-17T06:11:58.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:58.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:58.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:11:58.443 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:11:58.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:58.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:58.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:11:59.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:11:59.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:11:59.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:11:59.598 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:11:59.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:11:59.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:11:59.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:00.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:00.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:00.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:12:00.753 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:12:00.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:00.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:00.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:01.640 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:01.630+0000 7f9a67a818c0 -1 osd.0 35 log_to_monitors true 2026-02-17T06:12:01.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:01.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:01.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:12:01.909 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:12:01.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:01.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [5,35) [v2:127.0.0.1:6802/947630516,v1:127.0.0.1:6803/947630516] [v2:127.0.0.1:6804/947630516,v1:127.0.0.1:6805/947630516] exists,up 0f58482f-cd4e-42b5-a605-588ed4760dcd 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:12:02.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:12:02.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:12:02.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:12:02.058 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:12:02.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:12:02.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:12:02.058 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:12:02.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:12:02.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:12:02.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:12:02.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:12:02.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:12:02.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:12:02.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:12:02.328 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:12:02.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:12:02.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:02.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:12:02.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724546 2026-02-17T06:12:02.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724546 2026-02-17T06:12:02.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724546' 2026-02-17T06:12:02.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:02.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:12:02.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920773 2026-02-17T06:12:02.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920773 2026-02-17T06:12:02.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724546 1-141733920773' 2026-02-17T06:12:02.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:02.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724546 2026-02-17T06:12:02.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:02.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:12:02.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724546 2026-02-17T06:12:02.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:02.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724546 2026-02-17T06:12:02.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724546' 2026-02-17T06:12:02.476 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 167503724546 2026-02-17T06:12:02.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:02.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 167503724546 2026-02-17T06:12:02.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:12:03.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:12:03.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:03.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-17T06:12:03.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:03.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920773 2026-02-17T06:12:03.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:03.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:12:03.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920773 2026-02-17T06:12:03.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:03.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920773 2026-02-17T06:12:03.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920773' 2026-02-17T06:12:03.795 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 141733920773 2026-02-17T06:12:03.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:12:03.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920773 -lt 141733920773 2026-02-17T06:12:03.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:12:03.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:12:03.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:12:04.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:12:04.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:12:04.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:12:04.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:12:04.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:12:04.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:12:04.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:12:04.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:12:04.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:12:04.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:12:04.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:12:04.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:12:04.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:12:04.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:12:04.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:12:04.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5979: TEST_corrupt_snapset_scrub_rep: rm td/osd-scrub-repair/change 2026-02-17T06:12:04.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5981: TEST_corrupt_snapset_scrub_rep: pg_scrub 3.0 2026-02-17T06:12:04.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T06:12:04.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T06:12:04.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T06:12:04.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:12:04.476 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:12:04.476 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:12:04.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:12:04.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:12:04.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:12:04.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:12:04.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:12:04.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:12:04.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:12:04.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:12:04.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:12:04.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:12:04.936 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:12:04.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:12:04.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:04.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:12:05.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724547 2026-02-17T06:12:05.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724547 2026-02-17T06:12:05.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724547' 2026-02-17T06:12:05.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:05.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:12:05.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920774 2026-02-17T06:12:05.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920774 2026-02-17T06:12:05.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724547 1-141733920774' 2026-02-17T06:12:05.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:05.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724547 2026-02-17T06:12:05.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:05.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:12:05.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724547 2026-02-17T06:12:05.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:05.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724547 2026-02-17T06:12:05.082 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 167503724547 2026-02-17T06:12:05.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724547' 2026-02-17T06:12:05.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:05.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724547 2026-02-17T06:12:05.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:12:06.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:12:06.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:06.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-17T06:12:06.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:06.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920774 2026-02-17T06:12:06.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:06.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:12:06.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920774 2026-02-17T06:12:06.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:06.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920774 2026-02-17T06:12:06.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920774' 2026-02-17T06:12:06.404 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 141733920774 2026-02-17T06:12:06.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:12:06.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-17T06:12:06.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:12:06.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T06:12:06.562 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 3.0 loop 0 2026-02-17T06:12:06.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T06:12:06.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T06:12:06.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:12:06.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T06:12:06.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:06.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:06.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:06.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:06.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T06:12:06.930 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-17T06:12:06.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:06.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T06:12:06.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:06.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:12:06.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:12:06.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:12:06.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:12:06.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:06.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:06.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:06.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:07.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:38.785856+0000 '>' 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:07.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:12:08.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:12:08.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:12:08.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:12:08.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:08.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:08.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:08.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:08.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:38.785856+0000 '>' 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:08.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:12:09.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:12:09.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:12:09.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:12:09.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:09.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:09.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:09.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:09.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:38.785856+0000 '>' 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:09.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:12:10.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:12:10.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:12:10.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:12:10.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:10.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:10.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:10.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:10.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:38.785856+0000 '>' 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:10.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:12:11.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:12:11.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:12:11.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:12:11.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:11.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:11.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:11.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:11.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:38.785856+0000 '>' 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:11.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:12:12.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:12:12.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:12:12.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:12:12.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:12.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:12.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:12.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:12.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:11:38.785856+0000 '>' 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:12.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:12:13.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:12:13.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:12:13.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T06:12:13.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T06:12:13.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:12:13.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:12:13.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T06:12:14.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:12:07.600840+0000 '>' 2026-02-17T06:11:38.785856+0000 2026-02-17T06:12:14.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:12:14.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5983: TEST_corrupt_snapset_scrub_rep: rados list-inconsistent-pg csr_pool 2026-02-17T06:12:14.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5985: TEST_corrupt_snapset_scrub_rep: jq '. | length' td/osd-scrub-repair/json 2026-02-17T06:12:14.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5985: TEST_corrupt_snapset_scrub_rep: test 1 = 1 2026-02-17T06:12:14.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5987: TEST_corrupt_snapset_scrub_rep: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T06:12:14.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5987: TEST_corrupt_snapset_scrub_rep: test 3.0 = 3.0 2026-02-17T06:12:14.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5989: TEST_corrupt_snapset_scrub_rep: rados list-inconsistent-obj 3.0 2026-02-17T06:12:14.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5991: TEST_corrupt_snapset_scrub_rep: jq 'def walk(f): 2026-02-17T06:12:14.073 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:12:14.073 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:12:14.073 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:12:14.073 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:12:14.073 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:12:14.073 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5991: TEST_corrupt_snapset_scrub_rep: jq .inconsistents 2026-02-17T06:12:14.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5991: TEST_corrupt_snapset_scrub_rep: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:12:14.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6157: TEST_corrupt_snapset_scrub_rep: jq 'def walk(f): 2026-02-17T06:12:14.090 INFO:tasks.workunit.client.0.trial074.stderr: . as $in 2026-02-17T06:12:14.090 INFO:tasks.workunit.client.0.trial074.stderr: | if type == "object" then 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr: reduce keys[] as $key 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr: else f 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr: end; 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T06:12:14.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6157: TEST_corrupt_snapset_scrub_rep: jq .inconsistents 2026-02-17T06:12:14.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6157: TEST_corrupt_snapset_scrub_rep: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T06:12:14.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6158: TEST_corrupt_snapset_scrub_rep: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:12:14.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T06:12:14.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6159: TEST_corrupt_snapset_scrub_rep: test no = yes 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6164: TEST_corrupt_snapset_scrub_rep: test '' = yes 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6169: TEST_corrupt_snapset_scrub_rep: ERRORS=0 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6170: TEST_corrupt_snapset_scrub_rep: declare -a err_strings 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6171: TEST_corrupt_snapset_scrub_rep: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ1:head : snapset inconsistent' 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6172: TEST_corrupt_snapset_scrub_rep: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ2:head : snapset inconsistent' 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6173: TEST_corrupt_snapset_scrub_rep: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 [0-9]*:.*:::ROBJ1:1 : is an unexpected clone' 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6174: TEST_corrupt_snapset_scrub_rep: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6175: TEST_corrupt_snapset_scrub_rep: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 0 missing, 2 inconsistent objects' 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6176: TEST_corrupt_snapset_scrub_rep: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 4 errors' 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T06:12:14.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ1:head : snapset inconsistent' td/osd-scrub-repair/osd.1.log 2026-02-17T06:12:14.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T06:12:14.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ2:head : snapset inconsistent' td/osd-scrub-repair/osd.1.log 2026-02-17T06:12:14.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T06:12:14.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 [0-9]*:.*:::ROBJ1:1 : is an unexpected clone' td/osd-scrub-repair/osd.1.log 2026-02-17T06:12:14.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T06:12:14.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-17T06:12:14.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T06:12:14.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 0 missing, 2 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-17T06:12:14.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T06:12:14.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 4 errors' td/osd-scrub-repair/osd.1.log 2026-02-17T06:12:14.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6187: TEST_corrupt_snapset_scrub_rep: '[' 0 '!=' 0 ']' 2026-02-17T06:12:14.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6193: TEST_corrupt_snapset_scrub_rep: ceph osd pool rm csr_pool csr_pool --yes-i-really-really-mean-it 2026-02-17T06:12:14.323 INFO:tasks.workunit.client.0.trial074.stderr:pool 'csr_pool' removed 2026-02-17T06:12:14.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:12:14.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:12:14.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:12:14.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:12:14.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:12:14.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:12:14.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:12:14.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:12:14.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:12:14.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:12:14.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:12:14.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:12:14.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:12:14.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:12:14.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:12:14.654 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:12:14.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:12:14.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:12:14.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:12:14.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:12:14.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:12:14.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:12:14.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:12:14.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:12:14.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:12:14.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:12:14.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:12:14.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:12:14.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:12:14.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:12:14.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:12:14.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:12:14.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:12:14.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:12:14.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:12:14.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:12:14.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:12:14.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:12:14.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:12:14.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:12:14.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:12:14.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:12:14.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:12:14.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:12:14.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:12:14.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:12:14.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:12:14.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:12:14.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:12:14.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:12:14.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:12:14.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:12:14.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_list_missing_erasure_coded_appends td/osd-scrub-repair 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:978: TEST_list_missing_erasure_coded_appends: list_missing_erasure_coded td/osd-scrub-repair false 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:916: list_missing_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:917: list_missing_erasure_coded: local allow_overwrites=false 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: list_missing_erasure_coded: local poolname=ecpool 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:920: list_missing_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:12:14.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:12:14.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:12:14.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:12:14.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:12:14.713 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:12:14.713 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.713 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:14.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:12:14.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:12:14.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:12:14.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:12:14.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:12:14.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:12:14.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:12:14.746 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:12:14.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:12:14.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:12:14.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:12:14.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:12:14.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:12:14.803 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:12:14.803 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:12:14.803 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:12:14.804 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:12:14.804 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.804 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.804 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:12:14.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:12:14.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:12:14.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:921: list_missing_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T06:12:14.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:12:14.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:12:14.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:12:14.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:12:14.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:12:14.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:12:14.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:12:14.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:12:14.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:12:14.951 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:12:14.951 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.951 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:14.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:12:14.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:12:14.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: seq 0 2 2026-02-17T06:12:14.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:12:14.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:12:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:12:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:12:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:12:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:12:14.974 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:12:14.974 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:14.974 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:12:14.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:12:14.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:12:14.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ec4112c7-c588-41ff-8bd3-b7a16f52af24 2026-02-17T06:12:14.977 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 ec4112c7-c588-41ff-8bd3-b7a16f52af24 2026-02-17T06:12:14.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 ec4112c7-c588-41ff-8bd3-b7a16f52af24' 2026-02-17T06:12:14.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:12:14.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC+BpRpclyfOhAAyp1w+VnfDbM6DNqWQCtJBA== 2026-02-17T06:12:14.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC+BpRpclyfOhAAyp1w+VnfDbM6DNqWQCtJBA=="}' 2026-02-17T06:12:14.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ec4112c7-c588-41ff-8bd3-b7a16f52af24 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:12:15.078 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:12:15.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:12:15.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+BpRpclyfOhAAyp1w+VnfDbM6DNqWQCtJBA== --osd-uuid ec4112c7-c588-41ff-8bd3-b7a16f52af24 2026-02-17T06:12:15.102 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:15.094+0000 7f68d0dde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:15.104 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:15.098+0000 7f68d0dde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:15.105 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:15.098+0000 7f68d0dde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:15.106 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:15.098+0000 7f68d0dde8c0 -1 bdev(0x55e4b009fc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:12:15.106 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:15.098+0000 7f68d0dde8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:12:17.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:12:17.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:12:17.902 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:12:17.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:12:17.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:12:18.082 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:12:18.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:12:18.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:12:18.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:12:18.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:12:18.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:12:18.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:18.090+0000 7f7f09a3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:18.098 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:18.090+0000 7f7f09a3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:18.099 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:18.090+0000 7f7f09a3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:18.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:12:18.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:12:18.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:12:18.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:12:18.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:12:18.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:18.235 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:12:18.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:12:18.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:18.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:18.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:19.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:19.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:19.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:12:19.390 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:12:19.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:19.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:19.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:19.802 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:19.794+0000 7f7f09a3e8c0 -1 Falling back to public interface 2026-02-17T06:12:20.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:20.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:20.539 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:12:20.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:12:20.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:20.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:20.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:21.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:21.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:21.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:12:21.691 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:12:21.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:21.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:21.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:22.779 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:22.770+0000 7f7f09a3e8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:12:22.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:22.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:22.844 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:12:22.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:12:22.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:22.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:22.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:23.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:23.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:23.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:12:23.993 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:12:23.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:23.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:12:24.142 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1097559315,v1:127.0.0.1:6803/1097559315] [v2:127.0.0.1:6804/1097559315,v1:127.0.0.1:6805/1097559315] exists,up ec4112c7-c588-41ff-8bd3-b7a16f52af24 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:12:24.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:12:24.147 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:12:24.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:12:24.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:12:24.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:12:24.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:12:24.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e1739d05-5aa6-4116-a55c-e969ff804022 2026-02-17T06:12:24.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 e1739d05-5aa6-4116-a55c-e969ff804022' 2026-02-17T06:12:24.150 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 e1739d05-5aa6-4116-a55c-e969ff804022 2026-02-17T06:12:24.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:12:24.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDIBpRpqaNGCRAAqWDOMZ0NbZH7qzpkdRSflQ== 2026-02-17T06:12:24.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDIBpRpqaNGCRAAqWDOMZ0NbZH7qzpkdRSflQ=="}' 2026-02-17T06:12:24.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e1739d05-5aa6-4116-a55c-e969ff804022 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:12:24.309 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:12:24.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:12:24.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDIBpRpqaNGCRAAqWDOMZ0NbZH7qzpkdRSflQ== --osd-uuid e1739d05-5aa6-4116-a55c-e969ff804022 2026-02-17T06:12:24.334 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:24.326+0000 7f1c580a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:24.336 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:24.326+0000 7f1c580a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:24.337 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:24.330+0000 7f1c580a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:24.337 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:24.330+0000 7f1c580a48c0 -1 bdev(0x560376c2fc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:12:24.338 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:24.330+0000 7f1c580a48c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:12:26.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:12:26.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:12:26.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:12:26.642 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:12:26.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:12:26.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:12:26.831 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:12:26.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:12:26.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:12:26.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:12:26.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:12:26.854 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:26.846+0000 7f8fff4018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:26.856 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:26.846+0000 7f8fff4018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:26.857 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:26.850+0000 7f8fff4018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:12:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:12:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:12:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:12:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:12:26.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:26.988 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:12:26.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:12:26.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:26.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:12:27.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:28.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:28.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:28.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:12:28.139 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:12:28.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:28.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:12:28.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:28.562 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:28.554+0000 7f8fff4018c0 -1 Falling back to public interface 2026-02-17T06:12:29.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:29.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:29.292 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:12:29.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:12:29.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:29.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:12:29.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:30.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:30.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:30.444 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:12:30.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:12:30.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:30.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:12:30.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:31.521 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:31.514+0000 7f8fff4018c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:12:31.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:31.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:31.595 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:12:31.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:12:31.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:31.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:12:31.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:32.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:32.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:32.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:12:32.745 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:12:32.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:32.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:12:32.893 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4022901506,v1:127.0.0.1:6811/4022901506] [v2:127.0.0.1:6812/4022901506,v1:127.0.0.1:6813/4022901506] exists,up e1739d05-5aa6-4116-a55c-e969ff804022 2026-02-17T06:12:32.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:12:32.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:12:32.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:12:32.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:12:32.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T06:12:32.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:12:32.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:12:32.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:12:32.897 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:12:32.897 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:12:32.897 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:12:32.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:32.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:12:32.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:12:32.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:12:32.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:12:32.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:12:32.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a99f14ba-e85f-4de7-813b-8e14efec8e4e 2026-02-17T06:12:32.899 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 a99f14ba-e85f-4de7-813b-8e14efec8e4e 2026-02-17T06:12:32.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a99f14ba-e85f-4de7-813b-8e14efec8e4e' 2026-02-17T06:12:32.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:12:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDQBpRptNIYNhAAhIAxU+vCf4F8TRq8bbpFIQ== 2026-02-17T06:12:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDQBpRptNIYNhAAhIAxU+vCf4F8TRq8bbpFIQ=="}' 2026-02-17T06:12:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a99f14ba-e85f-4de7-813b-8e14efec8e4e -i td/osd-scrub-repair/2/new.json 2026-02-17T06:12:33.062 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:12:33.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T06:12:33.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDQBpRptNIYNhAAhIAxU+vCf4F8TRq8bbpFIQ== --osd-uuid a99f14ba-e85f-4de7-813b-8e14efec8e4e 2026-02-17T06:12:33.088 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:33.082+0000 7f7bcdc3b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:33.090 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:33.082+0000 7f7bcdc3b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:33.091 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:33.082+0000 7f7bcdc3b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:33.091 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:33.082+0000 7f7bcdc3b8c0 -1 bdev(0x5633c469fc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:12:33.091 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:33.082+0000 7f7bcdc3b8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T06:12:35.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T06:12:35.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:12:35.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:12:35.351 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:12:35.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:12:35.539 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:12:35.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:12:35.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:12:35.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:12:35.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:12:35.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:12:35.562 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:35.554+0000 7f696f6c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:35.564 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:35.558+0000 7f696f6c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:35.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:35.558+0000 7f696f6c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:12:35.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:12:35.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:12:35.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:12:35.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:12:35.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:12:35.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:35.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:12:35.692 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:12:35.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:35.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:12:35.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:36.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:36.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:36.846 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:12:36.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:12:36.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:36.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:12:36.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:37.045 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:37.038+0000 7f696f6c28c0 -1 Falling back to public interface 2026-02-17T06:12:37.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:37.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:37.996 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:12:37.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:12:37.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:37.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:12:38.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:39.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:39.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:39.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:12:39.149 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:12:39.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:39.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:12:39.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:40.007 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:12:39.998+0000 7f696f6c28c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:12:40.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:40.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:40.296 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:12:40.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:12:40.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:40.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:12:40.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:12:41.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:12:41.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:12:41.452 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:12:41.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:12:41.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:12:41.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:12:41.602 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2968078488,v1:127.0.0.1:6819/2968078488] [v2:127.0.0.1:6820/2968078488,v1:127.0.0.1:6821/2968078488] exists,up a99f14ba-e85f-4de7-813b-8e14efec8e4e 2026-02-17T06:12:41.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:12:41.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:12:41.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:12:41.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:925: list_missing_erasure_coded: create_rbd_pool 2026-02-17T06:12:41.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:12:41.741 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:12:41.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:12:41.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:12:41.946 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:12:41.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:12:42.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:12:43.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: list_missing_erasure_coded: wait_for_clean 2026-02-17T06:12:43.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:12:43.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:12:43.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:12:43.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:12:43.245 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:12:43.245 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:12:43.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:12:43.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:12:43.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:12:43.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:12:43.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:12:43.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:12:43.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:12:43.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:12:43.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:12:43.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:12:43.507 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:12:43.507 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:12:43.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:12:43.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:43.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:12:43.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T06:12:43.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T06:12:43.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T06:12:43.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:43.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:12:43.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T06:12:43.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T06:12:43.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-17T06:12:43.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:43.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:12:43.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T06:12:43.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T06:12:43.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-60129542146' 2026-02-17T06:12:43.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:43.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T06:12:43.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:43.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:12:43.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T06:12:43.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:43.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T06:12:43.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T06:12:43.726 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T06:12:43.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:43.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:12:43.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:12:44.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:12:44.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:45.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:12:45.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:12:46.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:12:46.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:46.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T06:12:46.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:46.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T06:12:46.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:46.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:12:46.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T06:12:46.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T06:12:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T06:12:46.202 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705668 2026-02-17T06:12:46.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:12:46.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T06:12:46.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:46.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T06:12:46.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:46.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:12:46.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T06:12:46.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:46.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T06:12:46.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T06:12:46.368 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542146 2026-02-17T06:12:46.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:12:46.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T06:12:46.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:12:46.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:12:46.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:12:46.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:12:46.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:12:46.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:12:46.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:12:46.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:12:46.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:12:46.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:12:46.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:12:46.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:12:46.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:12:46.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:12:46.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:12:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:12:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:12:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:12:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:928: list_missing_erasure_coded: create_ec_pool ecpool false k=2 m=1 2026-02-17T06:12:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T06:12:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T06:12:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T06:12:47.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T06:12:47.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T06:12:47.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T06:12:47.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T06:12:47.531 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T06:12:47.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:12:48.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T06:12:48.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T06:12:48.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:12:48.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:12:48.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:12:48.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:12:48.545 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:12:48.545 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:12:48.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:12:48.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:12:48.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:12:48.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:12:48.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:12:48.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:12:48.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:12:48.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:12:48.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:12:48.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:12:48.815 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:12:48.815 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:12:48.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:12:48.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:48.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:12:48.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T06:12:48.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T06:12:48.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T06:12:48.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:48.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:12:48.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T06:12:48.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T06:12:48.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670' 2026-02-17T06:12:48.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:12:48.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:12:49.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T06:12:49.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T06:12:49.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-60129542148' 2026-02-17T06:12:49.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:49.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T06:12:49.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:49.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:12:49.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T06:12:49.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:49.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T06:12:49.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T06:12:49.040 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T06:12:49.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:49.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T06:12:49.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:12:50.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:12:50.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:12:50.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-17T06:12:50.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:50.342 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T06:12:50.342 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:50.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:12:50.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T06:12:50.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:50.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T06:12:50.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T06:12:50.346 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705670 2026-02-17T06:12:50.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:12:50.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T06:12:50.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:12:50.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T06:12:50.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:12:50.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:12:50.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T06:12:50.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:12:50.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T06:12:50.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T06:12:50.503 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542148 2026-02-17T06:12:50.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:12:50.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T06:12:50.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:12:50.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:12:50.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:12:50.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T06:12:50.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:12:50.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:12:50.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:12:50.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:12:50.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:12:50.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:12:50.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:12:50.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T06:12:50.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:12:50.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:12:50.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:12:51.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T06:12:51.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:12:51.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:12:51.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T06:12:51.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:931: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ0 2026-02-17T06:12:51.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:12:51.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:12:51.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ0 2026-02-17T06:12:51.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:12:51.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:12:51.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:12:51.353 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:12:51.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:12:51.554 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:12:51.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:12:51.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:12:51.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ0 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:12:51.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: get_osds ecpool MOBJ0 2026-02-17T06:12:51.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T06:12:51.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ0 2026-02-17T06:12:51.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ0 2026-02-17T06:12:51.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:12:51.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T06:12:51.744 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:12:51.744 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:12:51.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T06:12:51.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: osds0=('2' '1' '0') 2026-02-17T06:12:51.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: local -a osds0 2026-02-17T06:12:51.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:935: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ1 2026-02-17T06:12:51.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:12:51.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:12:51.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ1 2026-02-17T06:12:51.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:12:51.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:12:51.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:12:51.939 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:12:51.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:12:52.143 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:12:52.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:12:52.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:12:52.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:12:52.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: get_osds ecpool MOBJ1 2026-02-17T06:12:52.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T06:12:52.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ1 2026-02-17T06:12:52.179 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ1 2026-02-17T06:12:52.179 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:12:52.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T06:12:52.330 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:12:52.330 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:12:52.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T06:12:52.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: osds1=('2' '1' '0') 2026-02-17T06:12:52.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: local -a osds1 2026-02-17T06:12:52.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: seq 0 2 2026-02-17T06:12:52.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:12:52.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:12:52.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:12:52.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:12:52.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:12:52.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:12:52.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:12:53.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:12:53.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:12:53.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:12:53.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:12:53.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:12:53.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:12:53.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:12:53.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:12:54.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:12:54.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:12:54.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:12:54.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:12:54.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:12:54.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:12:54.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:12:54.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:12:56.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:12:56.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:943: list_missing_erasure_coded: id=2 2026-02-17T06:12:56.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:944: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 MOBJ0 remove 2026-02-17T06:12:56.925 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#2:a9030505:::MOBJ0:head# 2026-02-17T06:12:57.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: list_missing_erasure_coded: id=1 2026-02-17T06:12:57.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:947: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ0 remove 2026-02-17T06:12:58.101 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:a9030505:::MOBJ0:head# 2026-02-17T06:12:58.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:950: list_missing_erasure_coded: id=1 2026-02-17T06:12:58.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:951: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ1 remove 2026-02-17T06:12:59.277 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:8dd16d55:::MOBJ1:head# 2026-02-17T06:12:59.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: list_missing_erasure_coded: id=0 2026-02-17T06:12:59.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:954: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 MOBJ1 remove 2026-02-17T06:13:00.445 INFO:tasks.workunit.client.0.trial074.stdout:remove 2#2:8dd16d55:::MOBJ1:head# 2026-02-17T06:13:00.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: seq 0 2 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 0 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:13:00.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:00.986 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:13:00.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:13:00.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:13:01.005 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:00.998+0000 7f3f05e9e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:01.007 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:00.998+0000 7f3f05e9e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:01.008 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:01.002+0000 7f3f05e9e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:01.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:13:01.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:13:01.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:13:01.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:13:01.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:13:01.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:01.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:13:01.142 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:13:01.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:01.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:01.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:02.281 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:02.274+0000 7f3f05e9e8c0 -1 Falling back to public interface 2026-02-17T06:13:02.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:02.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:02.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:13:02.295 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:13:02.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:02.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:02.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:03.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:03.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:03.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:13:03.446 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:13:03.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:03.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:03.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:04.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:04.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:04.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:13:04.601 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:13:04.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:04.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:04.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:05.250 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:05.242+0000 7f3f05e9e8c0 -1 osd.0 31 log_to_monitors true 2026-02-17T06:13:05.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:05.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:05.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:13:05.754 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:13:05.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:05.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:05.906 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 39 up_thru 39 down_at 32 last_clean_interval [5,31) [v2:127.0.0.1:6802/4247065849,v1:127.0.0.1:6803/4247065849] [v2:127.0.0.1:6804/4247065849,v1:127.0.0.1:6805/4247065849] exists,up ec4112c7-c588-41ff-8bd3-b7a16f52af24 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 1 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:05.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:05.908 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:05.908 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:05.908 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:05.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:13:05.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:13:05.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:13:05.910 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:13:05.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:13:05.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:13:05.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:13:05.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:13:05.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:13:05.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:13:05.928 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:05.922+0000 7f24093028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:05.929 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:05.922+0000 7f24093028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:05.930 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:05.922+0000 7f24093028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:06.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:13:06.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:13:06.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:13:06.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:13:06.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:13:06.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:06.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:13:06.066 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:13:06.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:06.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:06.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:07.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:07.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:07.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:13:07.221 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:13:07.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:07.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:07.221 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:07.214+0000 7f24093028c0 -1 Falling back to public interface 2026-02-17T06:13:07.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:08.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:08.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:08.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:13:08.373 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:13:08.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:08.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:08.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:09.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:09.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:09.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:13:09.528 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:13:09.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:09.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:09.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:10.445 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:10.438+0000 7f24093028c0 -1 osd.1 33 log_to_monitors true 2026-02-17T06:13:10.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:10.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:10.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:13:10.680 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:13:10.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:10.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:10.834 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 43 up_thru 43 down_at 34 last_clean_interval [9,33) [v2:127.0.0.1:6810/754435503,v1:127.0.0.1:6811/754435503] [v2:127.0.0.1:6812/754435503,v1:127.0.0.1:6813/754435503] exists,up e1739d05-5aa6-4116-a55c-e969ff804022 2026-02-17T06:13:10.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:13:10.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:13:10.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:13:10.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 2 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:13:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:10.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:13:10.837 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:13:10.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:13:10.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:13:10.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:13:10.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:13:10.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:13:10.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:13:10.852 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:10.846+0000 7f6d11bc98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:10.854 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:10.846+0000 7f6d11bc98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:10.854 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:10.846+0000 7f6d11bc98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:10.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:13:10.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:11.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:12.085 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:12.078+0000 7f6d11bc98c0 -1 Falling back to public interface 2026-02-17T06:13:12.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:12.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:12.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:13:12.140 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:13:12.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:12.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:12.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:13.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:13.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:13.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:13:13.290 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:13:13.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:13.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:13.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:14.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:14.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:14.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:13:14.440 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:13:14.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:14.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:14.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:15.077 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:15.070+0000 7f6d11bc98c0 -1 osd.2 36 log_to_monitors true 2026-02-17T06:13:15.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:15.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:15.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:13:15.593 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:13:15.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:15.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:15.743 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 47 up_thru 47 down_at 36 last_clean_interval [14,36) [v2:127.0.0.1:6818/3379968172,v1:127.0.0.1:6819/3379968172] [v2:127.0.0.1:6820/3379968172,v1:127.0.0.1:6821/3379968172] exists,up a99f14ba-e85f-4de7-813b-8e14efec8e4e 2026-02-17T06:13:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:13:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:13:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:13:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:960: list_missing_erasure_coded: create_rbd_pool 2026-02-17T06:13:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:13:15.938 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' removed 2026-02-17T06:13:15.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:13:15.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:13:16.143 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:13:16.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:13:17.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:13:17.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:961: list_missing_erasure_coded: wait_for_clean 2026-02-17T06:13:17.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:13:17.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:13:17.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:13:17.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:13:17.437 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:13:17.437 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:13:17.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:13:17.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:13:17.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:13:17.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:13:17.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:13:17.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:13:17.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:13:17.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:13:17.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:13:17.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:13:17.706 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:13:17.706 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:13:17.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:13:17.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:17.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:13:17.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724548 2026-02-17T06:13:17.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724548 2026-02-17T06:13:17.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724548' 2026-02-17T06:13:17.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:17.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:13:17.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593731 2026-02-17T06:13:17.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593731 2026-02-17T06:13:17.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724548 1-184683593731' 2026-02-17T06:13:17.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:17.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:13:17.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=201863462914 2026-02-17T06:13:17.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 201863462914 2026-02-17T06:13:17.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724548 1-184683593731 2-201863462914' 2026-02-17T06:13:17.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:13:17.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724548 2026-02-17T06:13:17.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:13:17.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:13:17.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724548 2026-02-17T06:13:17.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:13:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724548 2026-02-17T06:13:17.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724548' 2026-02-17T06:13:17.927 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 167503724548 2026-02-17T06:13:17.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:13:18.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724547 -lt 167503724548 2026-02-17T06:13:18.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:13:19.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:13:19.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:13:19.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724547 -lt 167503724548 2026-02-17T06:13:19.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:13:20.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:13:20.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:13:20.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724548 -lt 167503724548 2026-02-17T06:13:20.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:13:20.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-184683593731 2026-02-17T06:13:20.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:13:20.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:13:20.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-184683593731 2026-02-17T06:13:20.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:13:20.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593731 2026-02-17T06:13:20.393 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 184683593731 2026-02-17T06:13:20.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 184683593731' 2026-02-17T06:13:20.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:13:20.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-17T06:13:20.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:13:20.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-201863462914 2026-02-17T06:13:20.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:13:20.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:13:20.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-201863462914 2026-02-17T06:13:20.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:13:20.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=201863462914 2026-02-17T06:13:20.561 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 201863462914 2026-02-17T06:13:20.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 201863462914' 2026-02-17T06:13:20.562 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:13:20.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 201863462914 -lt 201863462914 2026-02-17T06:13:20.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:13:20.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:13:20.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:13:20.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T06:13:20.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:13:20.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:13:20.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:13:20.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:13:20.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:13:20.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:13:20.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:13:21.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T06:13:21.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:13:21.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:13:21.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:13:21.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T06:13:21.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:13:21.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:13:21.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: get_pg ecpool MOBJ0 2026-02-17T06:13:21.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T06:13:21.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=MOBJ0 2026-02-17T06:13:21.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool MOBJ0 2026-02-17T06:13:21.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:13:21.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: local pg=2.0 2026-02-17T06:13:21.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:968: list_missing_erasure_coded: repair 2.0 2026-02-17T06:13:21.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T06:13:21.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T06:13:21.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:13:21.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:13:21.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:13:21.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:13:21.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:12:47.527328+0000 2026-02-17T06:13:21.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T06:13:21.669 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.2 to repair 2026-02-17T06:13:21.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T06:12:47.527328+0000 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:12:47.527328+0000 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:13:21.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:13:21.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:13:21.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:13:21.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:12:47.527328+0000 '>' 2026-02-17T06:12:47.527328+0000 2026-02-17T06:13:21.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:13:22.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:13:22.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:13:22.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:13:22.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:13:22.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:13:22.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:13:22.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:13:22.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:12:47.527328+0000 '>' 2026-02-17T06:12:47.527328+0000 2026-02-17T06:13:22.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:13:23.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:13:23.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:13:23.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:13:23.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:13:23.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:13:23.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:13:23.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:13:24.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:12:47.527328+0000 '>' 2026-02-17T06:12:47.527328+0000 2026-02-17T06:13:24.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:13:25.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:13:25.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:13:25.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:13:25.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:13:25.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:13:25.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:13:25.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:13:25.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:13:22.043173+0000 '>' 2026-02-17T06:12:47.527328+0000 2026-02-17T06:13:25.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:13:25.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: seq 0 120 2026-02-17T06:13:25.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: for i in $(seq 0 120) 2026-02-17T06:13:25.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:971: list_missing_erasure_coded: '[' 0 -lt 60 ']' 2026-02-17T06:13:25.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: ceph pg 2.0 list_unfound 2026-02-17T06:13:25.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: egrep 'MOBJ0|MOBJ1' 2026-02-17T06:13:25.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: wc -l 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: matches=2 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: '[' 2 -eq 2 ']' 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: break 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:13:25.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:13:25.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:13:25.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:13:25.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:13:25.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:13:25.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:13:25.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:13:25.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:13:25.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:13:25.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:13:25.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:13:25.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:13:25.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:13:25.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:13:25.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:13:25.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:13:25.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:13:25.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:13:25.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:13:25.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:13:25.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:13:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:13:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:13:25.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:13:25.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:13:25.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:13:25.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:13:25.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:13:25.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:13:25.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:13:25.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:13:25.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:13:25.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:13:25.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:13:25.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:13:25.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:13:25.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:13:25.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:13:25.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:13:25.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:13:25.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:13:25.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:13:25.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:13:25.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:13:25.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:13:25.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:13:25.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:13:25.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:13:25.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:13:25.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:13:25.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:13:25.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:13:25.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_list_missing_erasure_coded_overwrites td/osd-scrub-repair 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: TEST_list_missing_erasure_coded_overwrites: '[' true = true ']' 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:983: TEST_list_missing_erasure_coded_overwrites: list_missing_erasure_coded td/osd-scrub-repair true 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:916: list_missing_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:917: list_missing_erasure_coded: local allow_overwrites=true 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: list_missing_erasure_coded: local poolname=ecpool 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:920: list_missing_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:13:25.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:13:25.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:13:25.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:25.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:25.539 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:25.539 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.539 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:25.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:13:25.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:13:25.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:13:25.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:13:25.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:13:25.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:13:25.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:13:25.572 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:13:25.572 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:13:25.572 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:13:25.572 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:13:25.572 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.572 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.572 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:13:25.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:13:25.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:13:25.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:13:25.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:13:25.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:13:25.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:13:25.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:13:25.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:13:25.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:921: list_missing_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T06:13:25.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:13:25.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:13:25.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:13:25.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:13:25.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:13:25.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:13:25.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:13:25.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:25.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:25.779 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:25.779 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.779 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:25.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:13:25.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:13:25.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: seq 0 2 2026-02-17T06:13:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:13:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T06:13:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:13:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:13:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:13:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:13:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:13:25.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:13:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:13:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:13:25.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:13:25.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c12a46c1-034b-4f72-b788-2704126e5947 2026-02-17T06:13:25.808 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 c12a46c1-034b-4f72-b788-2704126e5947 2026-02-17T06:13:25.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 c12a46c1-034b-4f72-b788-2704126e5947' 2026-02-17T06:13:25.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:13:25.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAFB5RpC8d+MBAA/ZJNuWxH/7v1oOroxo799w== 2026-02-17T06:13:25.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAFB5RpC8d+MBAA/ZJNuWxH/7v1oOroxo799w=="}' 2026-02-17T06:13:25.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c12a46c1-034b-4f72-b788-2704126e5947 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:13:25.905 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:13:25.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:13:25.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAFB5RpC8d+MBAA/ZJNuWxH/7v1oOroxo799w== --osd-uuid c12a46c1-034b-4f72-b788-2704126e5947 2026-02-17T06:13:25.930 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:25.922+0000 7effdb4bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:25.931 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:25.922+0000 7effdb4bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:25.932 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:25.926+0000 7effdb4bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:25.933 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:25.926+0000 7effdb4bf8c0 -1 bdev(0x55e785fe7c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:13:25.933 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:25.926+0000 7effdb4bf8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:13:28.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:13:28.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:13:28.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:13:28.566 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:13:28.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:13:28.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:13:28.763 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:13:28.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:13:28.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:13:28.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:13:28.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:13:28.786 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:28.778+0000 7f71807c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:28.788 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:28.782+0000 7f71807c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:28.789 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:28.782+0000 7f71807c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:28.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:13:28.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:13:28.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:13:28.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:13:28.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:13:28.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:28.914 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:13:28.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:13:28.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:28.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:29.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:30.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:30.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:30.067 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:13:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:13:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:30.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:30.281 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:30.274+0000 7f71807c78c0 -1 Falling back to public interface 2026-02-17T06:13:31.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:31.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:31.220 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:13:31.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:13:31.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:31.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:31.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:32.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:32.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:32.375 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:13:32.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:13:32.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:32.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:32.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:33.305 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:33.298+0000 7f71807c78c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:13:33.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:33.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:33.530 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:13:33.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:13:33.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:33.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:33.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:34.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:34.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:34.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:13:34.684 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:13:34.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:34.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:13:34.837 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/865818382,v1:127.0.0.1:6803/865818382] [v2:127.0.0.1:6804/865818382,v1:127.0.0.1:6805/865818382] exists,up c12a46c1-034b-4f72-b788-2704126e5947 2026-02-17T06:13:34.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:13:34.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:13:34.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:13:34.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:13:34.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T06:13:34.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:13:34.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:13:34.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:13:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:13:34.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:13:34.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=38aaf4a2-c381-43b4-a90b-8b6865037550 2026-02-17T06:13:34.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 38aaf4a2-c381-43b4-a90b-8b6865037550' 2026-02-17T06:13:34.846 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 38aaf4a2-c381-43b4-a90b-8b6865037550 2026-02-17T06:13:34.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:13:34.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAOB5RpqjTBMhAAZ8gOEIKY8k2Op9AAKYjXYA== 2026-02-17T06:13:34.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAOB5RpqjTBMhAAZ8gOEIKY8k2Op9AAKYjXYA=="}' 2026-02-17T06:13:34.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 38aaf4a2-c381-43b4-a90b-8b6865037550 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:13:35.012 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:13:35.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:13:35.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAOB5RpqjTBMhAAZ8gOEIKY8k2Op9AAKYjXYA== --osd-uuid 38aaf4a2-c381-43b4-a90b-8b6865037550 2026-02-17T06:13:35.038 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:35.030+0000 7f0a2ff778c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:35.039 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:35.034+0000 7f0a2ff778c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:35.040 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:35.034+0000 7f0a2ff778c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:35.041 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:35.034+0000 7f0a2ff778c0 -1 bdev(0x5565fd503c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:13:35.041 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:35.034+0000 7f0a2ff778c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:13:37.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:13:37.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:13:37.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:13:37.321 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:13:37.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:13:37.515 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:13:37.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:13:37.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:13:37.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:13:37.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:13:37.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:13:37.539 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:37.530+0000 7fde5d4428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:37.541 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:37.534+0000 7fde5d4428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:37.542 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:37.534+0000 7fde5d4428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:37.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:13:37.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:13:37.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:13:37.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:13:37.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:13:37.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:37.665 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:13:37.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:13:37.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:37.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:37.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:38.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:38.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:38.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:13:38.818 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:13:38.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:38.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:38.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:39.057 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:39.050+0000 7fde5d4428c0 -1 Falling back to public interface 2026-02-17T06:13:39.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:39.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:39.974 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:13:39.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:13:39.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:39.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:40.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:41.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:41.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:41.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:13:41.127 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:13:41.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:41.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:41.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:42.032 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:42.026+0000 7fde5d4428c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:13:42.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:42.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:42.281 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:13:42.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:13:42.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:42.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:42.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:43.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:43.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:43.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:13:43.436 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:13:43.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:43.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:13:43.588 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2629836864,v1:127.0.0.1:6811/2629836864] [v2:127.0.0.1:6812/2629836864,v1:127.0.0.1:6813/2629836864] exists,up 38aaf4a2-c381-43b4-a90b-8b6865037550 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:13:43.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:13:43.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:13:43.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:13:43.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:13:43.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:13:43.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:13:43.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:13:43.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:13:43.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:13:43.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a5ffd395-fd88-46cc-b315-e2287c9479c6 2026-02-17T06:13:43.597 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 a5ffd395-fd88-46cc-b315-e2287c9479c6 2026-02-17T06:13:43.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a5ffd395-fd88-46cc-b315-e2287c9479c6' 2026-02-17T06:13:43.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:13:43.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAXB5RpEEf3IxAALF+k/eXWigyLQjKtpTjLwQ== 2026-02-17T06:13:43.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAXB5RpEEf3IxAALF+k/eXWigyLQjKtpTjLwQ=="}' 2026-02-17T06:13:43.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a5ffd395-fd88-46cc-b315-e2287c9479c6 -i td/osd-scrub-repair/2/new.json 2026-02-17T06:13:43.762 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:13:43.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T06:13:43.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAXB5RpEEf3IxAALF+k/eXWigyLQjKtpTjLwQ== --osd-uuid a5ffd395-fd88-46cc-b315-e2287c9479c6 2026-02-17T06:13:43.789 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:43.782+0000 7f161016f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:43.790 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:43.782+0000 7f161016f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:43.791 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:43.782+0000 7f161016f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:43.792 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:43.786+0000 7f161016f8c0 -1 bdev(0x557701e5bc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:13:43.792 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:43.786+0000 7f161016f8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T06:13:46.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T06:13:46.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:13:46.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:13:46.082 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:13:46.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:13:46.277 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:13:46.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:13:46.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:13:46.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:13:46.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:13:46.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:13:46.301 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:46.294+0000 7f9c3f9e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:46.303 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:46.294+0000 7f9c3f9e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:46.304 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:46.298+0000 7f9c3f9e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:13:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:13:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:13:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:13:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:13:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:13:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:46.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:13:46.429 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:13:46.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:46.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:46.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:47.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:47.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:47.581 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:13:47.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:13:47.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:47.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:47.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:47.793 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:47.786+0000 7f9c3f9e28c0 -1 Falling back to public interface 2026-02-17T06:13:48.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:48.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:48.733 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:13:48.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:13:48.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:48.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:48.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:49.887 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:13:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:13:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:50.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:51.003 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:13:50.994+0000 7f9c3f9e28c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:13:51.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:51.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:51.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:13:51.040 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:13:51.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:51.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:51.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:13:52.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:13:52.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:13:52.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:13:52.196 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:13:52.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:13:52.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:13:52.345 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2186110746,v1:127.0.0.1:6819/2186110746] [v2:127.0.0.1:6820/2186110746,v1:127.0.0.1:6821/2186110746] exists,up a5ffd395-fd88-46cc-b315-e2287c9479c6 2026-02-17T06:13:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:13:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:13:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:13:52.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:925: list_missing_erasure_coded: create_rbd_pool 2026-02-17T06:13:52.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:13:52.486 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:13:52.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:13:52.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:13:52.709 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:13:52.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:13:53.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:13:54.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: list_missing_erasure_coded: wait_for_clean 2026-02-17T06:13:54.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:13:54.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:13:54.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:13:54.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:13:54.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:13:54.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:13:54.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:13:54.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:13:54.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:13:54.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:13:54.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:13:54.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:13:54.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:13:54.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:13:54.124 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:13:54.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:13:54.274 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:13:54.274 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:13:54.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:13:54.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:54.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:13:54.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T06:13:54.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T06:13:54.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T06:13:54.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:54.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:13:54.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T06:13:54.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T06:13:54.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-17T06:13:54.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:54.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:13:54.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574850 2026-02-17T06:13:54.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574850 2026-02-17T06:13:54.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-55834574850' 2026-02-17T06:13:54.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:13:54.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T06:13:54.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:13:54.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:13:54.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T06:13:54.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:13:54.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T06:13:54.493 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T06:13:54.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T06:13:54.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:13:54.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:13:54.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:13:55.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:13:55.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:13:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:13:55.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:13:56.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:13:56.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:13:56.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T06:13:56.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:13:56.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T06:13:56.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:13:56.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:13:56.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T06:13:56.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:13:56.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T06:13:56.966 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705668 2026-02-17T06:13:56.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T06:13:56.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:13:57.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T06:13:57.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:13:57.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574850 2026-02-17T06:13:57.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:13:57.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:13:57.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574850 2026-02-17T06:13:57.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:13:57.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574850 2026-02-17T06:13:57.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574850' 2026-02-17T06:13:57.130 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574850 2026-02-17T06:13:57.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:13:57.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574850 -lt 55834574850 2026-02-17T06:13:57.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:13:57.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:13:57.290 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:13:57.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:13:57.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:13:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:13:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:13:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:13:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:13:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:13:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:13:57.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:13:57.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:13:57.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:13:57.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:928: list_missing_erasure_coded: create_ec_pool ecpool true k=2 m=1 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T06:13:57.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T06:13:58.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T06:13:58.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T06:13:58.309 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T06:13:58.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:13:59.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T06:13:59.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T06:13:59.487 INFO:tasks.workunit.client.0.trial074.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T06:13:59.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T06:13:59.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:13:59.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:13:59.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:13:59.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:13:59.497 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:13:59.497 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:13:59.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:13:59.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:13:59.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:13:59.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:13:59.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:13:59.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:13:59.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:13:59.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:13:59.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:13:59.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:13:59.771 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:13:59.772 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:13:59.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:13:59.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:59.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:13:59.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T06:13:59.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T06:13:59.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T06:13:59.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:59.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:13:59.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T06:13:59.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T06:13:59.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670' 2026-02-17T06:13:59.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:13:59.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:13:59.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-17T06:13:59.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-17T06:13:59.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670 2-55834574852' 2026-02-17T06:13:59.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:13:59.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T06:13:59.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:13:59.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:13:59.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T06:13:59.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:13:59.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T06:13:59.991 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836488 2026-02-17T06:13:59.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T06:13:59.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:14:00.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T06:14:00.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:00.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T06:14:00.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:00.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:14:00.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T06:14:00.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:00.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T06:14:00.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T06:14:00.157 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705670 2026-02-17T06:14:00.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:14:00.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T06:14:00.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:00.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-17T06:14:00.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:00.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:14:00.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-17T06:14:00.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:00.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-17T06:14:00.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-17T06:14:00.316 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574852 2026-02-17T06:14:00.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:14:00.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-17T06:14:00.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:14:00.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:14:00.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:14:00.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:14:00.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T06:14:00.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:14:00.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:14:00.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:14:00.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T06:14:00.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:14:00.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:14:00.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T06:14:00.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:931: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ0 2026-02-17T06:14:00.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:14:00.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:14:00.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ0 2026-02-17T06:14:00.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:14:00.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:14:00.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:14:01.179 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:14:01.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:14:01.382 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:14:01.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:14:01.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:14:01.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ0 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:14:01.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: get_osds ecpool MOBJ0 2026-02-17T06:14:01.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T06:14:01.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ0 2026-02-17T06:14:01.420 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ0 2026-02-17T06:14:01.420 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:14:01.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T06:14:01.574 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:14:01.574 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:14:01.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: osds0=('2' '1' '0') 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: local -a osds0 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:935: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ1 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ1 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:14:01.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:14:01.767 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:14:01.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:14:01.973 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:14:01.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:14:01.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:14:01.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:14:02.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: get_osds ecpool MOBJ1 2026-02-17T06:14:02.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T06:14:02.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ1 2026-02-17T06:14:02.006 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ1 2026-02-17T06:14:02.006 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:14:02.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T06:14:02.157 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:14:02.157 INFO:tasks.workunit.client.0.trial074.stderr:0' 2026-02-17T06:14:02.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T06:14:02.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: osds1=('2' '1' '0') 2026-02-17T06:14:02.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: local -a osds1 2026-02-17T06:14:02.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: seq 0 2 2026-02-17T06:14:02.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:14:02.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:14:02.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:14:02.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:14:02.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:14:02.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:14:02.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:14:03.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:14:03.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:14:03.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:14:03.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:14:03.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:14:03.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:14:03.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:14:03.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:14:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:14:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:14:04.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:14:04.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:14:04.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:14:04.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:14:04.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:14:04.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:14:06.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:14:06.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:943: list_missing_erasure_coded: id=2 2026-02-17T06:14:06.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:944: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 MOBJ0 remove 2026-02-17T06:14:06.736 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#2:a9030505:::MOBJ0:head# 2026-02-17T06:14:07.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: list_missing_erasure_coded: id=1 2026-02-17T06:14:07.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:947: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ0 remove 2026-02-17T06:14:07.912 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:a9030505:::MOBJ0:head# 2026-02-17T06:14:08.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:950: list_missing_erasure_coded: id=1 2026-02-17T06:14:08.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:951: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ1 remove 2026-02-17T06:14:09.092 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#2:8dd16d55:::MOBJ1:head# 2026-02-17T06:14:09.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: list_missing_erasure_coded: id=0 2026-02-17T06:14:09.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:954: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 MOBJ1 remove 2026-02-17T06:14:10.272 INFO:tasks.workunit.client.0.trial074.stdout:remove 2#2:8dd16d55:::MOBJ1:head# 2026-02-17T06:14:10.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: seq 0 2 2026-02-17T06:14:10.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:14:10.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 0 2026-02-17T06:14:10.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:14:10.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:14:10.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:14:10.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:14:10.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:14:10.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:14:10.815 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:14:10.815 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:10.815 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:14:10.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:14:10.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:14:10.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:14:10.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:14:10.818 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:14:10.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:14:10.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:14:10.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:14:10.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:14:10.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:14:10.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:14:10.834 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:10.826+0000 7f14890608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:10.835 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:10.830+0000 7f14890608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:10.836 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:10.830+0000 7f14890608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:10.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:14:10.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:14:10.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:14:10.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:14:10.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:14:10.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:10.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:14:10.971 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:14:10.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:10.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:11.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:12.101 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:12.094+0000 7f14890608c0 -1 Falling back to public interface 2026-02-17T06:14:12.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:12.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:12.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:14:12.123 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:14:12.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:12.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:12.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:13.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:13.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:13.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:14:13.274 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:14:13.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:13.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:13.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:14.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:14.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:14.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:14:14.428 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:14:14.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:14.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:14.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:15.069 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:15.062+0000 7f14890608c0 -1 osd.0 31 log_to_monitors true 2026-02-17T06:14:15.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:15.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:15.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:14:15.581 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:14:15.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:15.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:15.739 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 38 up_thru 38 down_at 32 last_clean_interval [5,31) [v2:127.0.0.1:6802/3455267853,v1:127.0.0.1:6803/3455267853] [v2:127.0.0.1:6804/3455267853,v1:127.0.0.1:6805/3455267853] exists,up c12a46c1-034b-4f72-b788-2704126e5947 2026-02-17T06:14:15.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:14:15.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:14:15.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:14:15.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:14:15.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 1 2026-02-17T06:14:15.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:14:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:14:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:14:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:14:15.743 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:14:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:14:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:14:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:14:15.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:14:15.756 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:15.750+0000 7fccc1fff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:15.757 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:15.750+0000 7fccc1fff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:15.758 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:15.750+0000 7fccc1fff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:15.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:14:15.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:14:15.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:14:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:14:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:14:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:14:15.891 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:14:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:16.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:17.029 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:17.022+0000 7fccc1fff8c0 -1 Falling back to public interface 2026-02-17T06:14:17.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:17.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:17.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:14:17.044 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:14:17.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:17.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:17.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:18.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:18.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:18.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:14:18.198 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:14:18.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:18.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:18.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:19.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:19.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:19.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:14:19.352 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:14:19.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:19.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:19.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:20.240 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:20.234+0000 7fccc1fff8c0 -1 osd.1 33 log_to_monitors true 2026-02-17T06:14:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:20.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:14:20.504 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:14:20.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:20.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:20.656 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 42 up_thru 42 down_at 34 last_clean_interval [9,33) [v2:127.0.0.1:6810/126730842,v1:127.0.0.1:6811/126730842] [v2:127.0.0.1:6812/126730842,v1:127.0.0.1:6813/126730842] exists,up 38aaf4a2-c381-43b4-a90b-8b6865037550 2026-02-17T06:14:20.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:14:20.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 2 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:14:20.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:20.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:20.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:14:20.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:14:20.662 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:14:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:14:20.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:14:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:14:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:14:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:14:20.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:14:20.676 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:20.670+0000 7ff9c13998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:20.678 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:20.670+0000 7ff9c13998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:20.679 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:20.670+0000 7ff9c13998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:20.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:20.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:14:20.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:21.950 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:21.942+0000 7ff9c13998c0 -1 Falling back to public interface 2026-02-17T06:14:21.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:21.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:21.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:14:21.969 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:14:21.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:21.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:14:22.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:23.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:23.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:14:23.120 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:14:23.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:23.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:14:23.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:24.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:24.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:24.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:14:24.271 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:14:24.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:24.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:14:24.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:24.994 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:24.986+0000 7ff9c13998c0 -1 osd.2 35 log_to_monitors true 2026-02-17T06:14:25.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:25.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:25.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:14:25.425 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:14:25.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:25.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:14:25.575 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 47 up_thru 47 down_at 36 last_clean_interval [13,35) [v2:127.0.0.1:6818/2816715467,v1:127.0.0.1:6819/2816715467] [v2:127.0.0.1:6820/2816715467,v1:127.0.0.1:6821/2816715467] exists,up a5ffd395-fd88-46cc-b315-e2287c9479c6 2026-02-17T06:14:25.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:14:25.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:14:25.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:14:25.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:960: list_missing_erasure_coded: create_rbd_pool 2026-02-17T06:14:25.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:14:25.769 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' removed 2026-02-17T06:14:25.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:14:25.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:14:25.974 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:14:25.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:14:26.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:14:27.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:961: list_missing_erasure_coded: wait_for_clean 2026-02-17T06:14:27.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:14:27.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:14:27.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:14:27.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:14:27.269 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:14:27.269 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:14:27.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:14:27.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:14:27.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:14:27.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:14:27.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:14:27.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:14:27.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:14:27.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:14:27.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:14:27.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:14:27.539 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:14:27.539 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:14:27.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:14:27.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:14:27.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:14:27.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757252 2026-02-17T06:14:27.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757252 2026-02-17T06:14:27.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-163208757252' 2026-02-17T06:14:27.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:14:27.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:14:27.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626435 2026-02-17T06:14:27.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626435 2026-02-17T06:14:27.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-163208757252 1-180388626435' 2026-02-17T06:14:27.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:14:27.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:14:27.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=201863462914 2026-02-17T06:14:27.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 201863462914 2026-02-17T06:14:27.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-163208757252 1-180388626435 2-201863462914' 2026-02-17T06:14:27.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:27.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-163208757252 2026-02-17T06:14:27.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:27.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:14:27.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-163208757252 2026-02-17T06:14:27.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:27.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757252 2026-02-17T06:14:27.758 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 163208757252 2026-02-17T06:14:27.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 163208757252' 2026-02-17T06:14:27.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:14:27.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757251 -lt 163208757252 2026-02-17T06:14:27.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:14:28.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:14:28.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:14:29.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757252 -lt 163208757252 2026-02-17T06:14:29.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:29.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-180388626435 2026-02-17T06:14:29.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:29.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:14:29.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-180388626435 2026-02-17T06:14:29.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:29.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626435 2026-02-17T06:14:29.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 180388626435' 2026-02-17T06:14:29.073 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 180388626435 2026-02-17T06:14:29.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:14:29.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-17T06:14:29.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:29.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-201863462914 2026-02-17T06:14:29.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:29.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:14:29.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-201863462914 2026-02-17T06:14:29.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:29.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=201863462914 2026-02-17T06:14:29.236 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 201863462914 2026-02-17T06:14:29.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 201863462914' 2026-02-17T06:14:29.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:14:29.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 201863462914 -lt 201863462914 2026-02-17T06:14:29.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:14:29.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:14:29.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:14:29.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T06:14:29.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:14:29.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:14:29.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:14:29.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:14:29.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:14:29.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:14:29.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:14:29.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T06:14:29.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:14:29.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:14:29.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:14:29.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T06:14:29.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:14:29.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:14:29.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: get_pg ecpool MOBJ0 2026-02-17T06:14:29.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T06:14:29.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=MOBJ0 2026-02-17T06:14:29.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool MOBJ0 2026-02-17T06:14:29.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:14:30.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: local pg=2.0 2026-02-17T06:14:30.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:968: list_missing_erasure_coded: repair 2.0 2026-02-17T06:14:30.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T06:14:30.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T06:14:30.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:14:30.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:14:30.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:14:30.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:14:30.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:30.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T06:14:30.353 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0s0 on osd.2 to repair 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:14:30.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:14:30.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:14:30.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:13:58.305014+0000 '>' 2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:30.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:14:31.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:14:31.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:14:31.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:14:31.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:14:31.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:14:31.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:14:31.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:14:31.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:13:58.305014+0000 '>' 2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:31.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:14:32.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:14:32.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:14:32.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:14:32.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:14:32.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:14:32.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:14:32.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:14:32.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:13:58.305014+0000 '>' 2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:32.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:14:33.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:14:33.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:14:33.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:14:33.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:14:33.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:14:33.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:14:33.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:14:33.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:13:58.305014+0000 '>' 2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:33.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:14:34.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:14:34.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:14:34.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:14:34.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:14:34.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:14:34.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:14:34.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:14:35.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:13:58.305014+0000 '>' 2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:35.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:14:36.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:14:36.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:14:36.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:14:36.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:14:36.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:14:36.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:14:36.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:14:36.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:14:31.014682+0000 '>' 2026-02-17T06:13:58.305014+0000 2026-02-17T06:14:36.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:14:36.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: seq 0 120 2026-02-17T06:14:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: for i in $(seq 0 120) 2026-02-17T06:14:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:971: list_missing_erasure_coded: '[' 0 -lt 60 ']' 2026-02-17T06:14:36.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: ceph pg 2.0 list_unfound 2026-02-17T06:14:36.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: egrep 'MOBJ0|MOBJ1' 2026-02-17T06:14:36.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: wc -l 2026-02-17T06:14:36.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: matches=2 2026-02-17T06:14:36.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: '[' 2 -eq 2 ']' 2026-02-17T06:14:36.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: break 2026-02-17T06:14:36.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:14:36.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:14:36.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:14:36.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:14:36.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:14:36.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:14:36.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:14:36.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:14:36.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:14:36.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:14:36.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:14:36.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:14:36.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:14:36.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:14:36.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:14:36.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:14:36.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:14:36.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:14:36.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:14:36.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:14:36.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:14:36.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:14:36.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:14:36.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:14:36.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:14:36.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:14:36.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:14:36.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:14:36.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:14:36.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:14:36.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:14:36.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:14:36.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:14:36.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:14:36.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:14:36.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:14:36.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:14:36.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:14:36.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:14:36.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:14:36.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:14:36.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:14:36.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:14:36.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:14:36.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:14:36.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:14:36.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:14:36.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:14:36.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:14:36.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:14:36.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:14:36.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:14:36.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:14:36.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:14:36.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_periodic_scrub_replicated td/osd-scrub-repair 2026-02-17T06:14:36.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5759: TEST_periodic_scrub_replicated: local dir=td/osd-scrub-repair 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5760: TEST_periodic_scrub_replicated: local poolname=psr_pool 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5761: TEST_periodic_scrub_replicated: local objname=POBJ 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5763: TEST_periodic_scrub_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:14:36.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T06:14:36.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:14:36.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:14:36.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:14:36.548 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:14:36.548 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.548 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:36.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T06:14:36.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:14:36.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:14:36.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:14:36.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:14:36.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:14:36.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:14:36.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:14:36.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:14:36.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:14:36.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:14:36.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:14:36.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:14:36.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:14:36.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:14:36.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5764: TEST_periodic_scrub_replicated: run_mgr td/osd-scrub-repair x 2026-02-17T06:14:36.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:14:36.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:14:36.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:14:36.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:14:36.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:14:36.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:14:36.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:14:36.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:14:36.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:14:36.789 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:14:36.789 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.789 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:36.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:14:36.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5765: TEST_periodic_scrub_replicated: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5766: TEST_periodic_scrub_replicated: ceph_osd_args+=--osd_scrub_backoff_ratio=0 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5767: TEST_periodic_scrub_replicated: run_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:14:36.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:14:36.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:14:36.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:14:36.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:14:36.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e021134a-7e18-45c4-a735-8a1535137cd3 2026-02-17T06:14:36.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 e021134a-7e18-45c4-a735-8a1535137cd3' 2026-02-17T06:14:36.816 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 e021134a-7e18-45c4-a735-8a1535137cd3 2026-02-17T06:14:36.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:14:36.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBMB5RpaZ3vMBAAD4QTupwgtkOnif22wRET7w== 2026-02-17T06:14:36.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBMB5RpaZ3vMBAAD4QTupwgtkOnif22wRET7w=="}' 2026-02-17T06:14:36.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e021134a-7e18-45c4-a735-8a1535137cd3 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:14:36.914 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:14:36.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:14:36.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --mkfs --key AQBMB5RpaZ3vMBAAD4QTupwgtkOnif22wRET7w== --osd-uuid e021134a-7e18-45c4-a735-8a1535137cd3 2026-02-17T06:14:36.938 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:36.930+0000 7fa2da8f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:36.940 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:36.934+0000 7fa2da8f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:36.941 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:36.934+0000 7fa2da8f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:36.941 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:36.934+0000 7fa2da8f08c0 -1 bdev(0x5582c88c1c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:14:36.942 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:36.934+0000 7fa2da8f08c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:14:39.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:14:39.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:14:39.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:14:39.539 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:14:39.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:14:39.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:14:39.734 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:14:39.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:14:39.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:14:39.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:14:39.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:14:39.758 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:39.750+0000 7f93b279b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:39.760 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:39.754+0000 7f93b279b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:39.761 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:39.754+0000 7f93b279b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:39.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:39.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:40.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:41.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:41.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:41.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:14:41.039 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:14:41.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:41.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:41.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:41.249 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:41.242+0000 7f93b279b8c0 -1 Falling back to public interface 2026-02-17T06:14:42.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:42.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:42.194 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:14:42.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:14:42.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:42.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:42.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:43.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:43.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:43.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:14:43.349 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:14:43.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:43.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:43.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:44.207 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:44.198+0000 7f93b279b8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:14:44.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:44.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:44.503 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:14:44.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:14:44.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:44.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:44.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:45.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:45.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:45.660 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:14:45.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:14:45.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:45.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:14:45.814 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/335380301,v1:127.0.0.1:6803/335380301] [v2:127.0.0.1:6804/335380301,v1:127.0.0.1:6805/335380301] exists,up e021134a-7e18-45c4-a735-8a1535137cd3 2026-02-17T06:14:45.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5768: TEST_periodic_scrub_replicated: run_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:14:45.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:14:45.819 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:14:45.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:14:45.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T06:14:45.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:14:45.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:14:45.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=01566805-5272-43ca-807b-fb59ea486e8b 2026-02-17T06:14:45.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 01566805-5272-43ca-807b-fb59ea486e8b' 2026-02-17T06:14:45.822 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 01566805-5272-43ca-807b-fb59ea486e8b 2026-02-17T06:14:45.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:14:45.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBVB5RpSYlUMRAAv4G26+PTde5K6bRljKJZIg== 2026-02-17T06:14:45.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBVB5RpSYlUMRAAv4G26+PTde5K6bRljKJZIg=="}' 2026-02-17T06:14:45.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 01566805-5272-43ca-807b-fb59ea486e8b -i td/osd-scrub-repair/1/new.json 2026-02-17T06:14:45.989 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:14:45.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:14:45.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --mkfs --key AQBVB5RpSYlUMRAAv4G26+PTde5K6bRljKJZIg== --osd-uuid 01566805-5272-43ca-807b-fb59ea486e8b 2026-02-17T06:14:46.016 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:46.010+0000 7f61f9f2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:46.018 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:46.010+0000 7f61f9f2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:46.019 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:46.010+0000 7f61f9f2f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:46.019 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:46.014+0000 7f61f9f2f8c0 -1 bdev(0x5578885f1c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:14:46.019 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:46.014+0000 7f61f9f2f8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:14:48.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:14:48.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:14:48.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:14:48.551 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:14:48.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:14:48.743 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:14:48.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:14:48.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:14:48.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:14:48.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:14:48.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:14:48.769 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:48.762+0000 7f73b0e9d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:48.770 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:48.762+0000 7f73b0e9d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:48.771 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:48.766+0000 7f73b0e9d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:14:48.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:14:48.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:14:48.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:14:48.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:14:48.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:14:48.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:48.896 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:14:48.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:14:48.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:48.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:49.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:50.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:50.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:50.047 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:14:50.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:14:50.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:50.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:50.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:50.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:50.250+0000 7f73b0e9d8c0 -1 Falling back to public interface 2026-02-17T06:14:51.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:51.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:51.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:14:51.202 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:14:51.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:51.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:51.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:52.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:52.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:52.358 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:14:52.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:14:52.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:52.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:52.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:53.291 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:14:53.281+0000 7f73b0e9d8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:14:53.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:53.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:53.511 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:14:53.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:14:53.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:53.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:53.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:14:54.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:14:54.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:14:54.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:14:54.667 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:14:54.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:14:54.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:14:54.815 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/477483969,v1:127.0.0.1:6811/477483969] [v2:127.0.0.1:6812/477483969,v1:127.0.0.1:6813/477483969] exists,up 01566805-5272-43ca-807b-fb59ea486e8b 2026-02-17T06:14:54.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:14:54.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:14:54.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:14:54.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5769: TEST_periodic_scrub_replicated: create_rbd_pool 2026-02-17T06:14:54.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:14:54.957 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:14:54.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:14:54.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:14:55.211 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:14:55.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:14:56.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:14:56.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5770: TEST_periodic_scrub_replicated: wait_for_clean 2026-02-17T06:14:56.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:14:56.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:14:56.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:14:56.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:14:56.507 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:14:56.507 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:14:56.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:14:56.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:14:56.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:14:56.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:14:56.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:14:56.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:14:56.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:14:56.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:14:56.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:14:56.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:14:56.775 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:14:56.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:14:56.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:14:56.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:14:56.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:14:56.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:14:56.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:14:56.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:14:56.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:14:56.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T06:14:56.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T06:14:56.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T06:14:56.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:56.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:14:56.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:56.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:14:56.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:14:56.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:56.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:14:56.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:14:56.923 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:14:56.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:14:57.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:14:57.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:57.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T06:14:57.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:57.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:14:57.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T06:14:57.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:57.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T06:14:57.091 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705666 2026-02-17T06:14:57.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T06:14:57.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:14:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T06:14:57.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:14:57.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:14:57.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:14:57.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:14:57.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:14:57.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:14:57.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:14:57.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:14:57.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:14:57.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:14:57.440 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:14:57.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:14:57.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:14:57.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:14:57.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:14:57.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:14:57.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:14:57.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:14:57.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5772: TEST_periodic_scrub_replicated: create_pool psr_pool 1 1 2026-02-17T06:14:57.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool 1 1 2026-02-17T06:14:57.974 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool' created 2026-02-17T06:14:57.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:14:58.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5773: TEST_periodic_scrub_replicated: wait_for_clean 2026-02-17T06:14:58.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:14:58.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:14:58.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:14:58.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:14:58.989 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:14:58.989 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:14:58.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:14:58.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:14:58.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:14:59.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:14:59.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:14:59.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:14:59.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:14:59.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:14:59.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:14:59.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:14:59.260 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:14:59.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:14:59.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:14:59.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:14:59.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T06:14:59.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T06:14:59.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T06:14:59.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:14:59.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:14:59.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705667 2026-02-17T06:14:59.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705667 2026-02-17T06:14:59.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705667' 2026-02-17T06:14:59.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:14:59.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T06:14:59.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:14:59.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:14:59.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T06:14:59.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:14:59.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T06:14:59.411 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T06:14:59.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T06:14:59.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:14:59.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:14:59.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:00.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:15:00.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:00.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:15:00.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:01.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:15:01.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:01.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T06:15:01.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:01.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705667 2026-02-17T06:15:01.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:01.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:15:01.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705667 2026-02-17T06:15:01.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:01.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705667 2026-02-17T06:15:01.883 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705667 2026-02-17T06:15:01.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-17T06:15:01.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:15:02.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-17T06:15:02.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:15:02.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:15:02.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:15:02.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T06:15:02.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:15:02.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:15:02.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:15:02.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:15:02.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:15:02.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:15:02.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:15:02.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T06:15:02.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:15:02.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:15:02.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:15:02.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T06:15:02.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5775: TEST_periodic_scrub_replicated: local osd=0 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5776: TEST_periodic_scrub_replicated: add_something td/osd-scrub-repair psr_pool POBJ scrub 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=psr_pool 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=POBJ 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=scrub 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' scrub = noscrub ']' 2026-02-17T06:15:02.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:81: add_something: ceph osd unset noscrub 2026-02-17T06:15:02.764 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is unset 2026-02-17T06:15:02.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:82: add_something: ceph osd unset nodeep-scrub 2026-02-17T06:15:02.969 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is unset 2026-02-17T06:15:02.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:15:02.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:15:02.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool psr_pool put POBJ td/osd-scrub-repair/ORIGINAL 2026-02-17T06:15:03.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5777: TEST_periodic_scrub_replicated: get_primary psr_pool POBJ 2026-02-17T06:15:03.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=psr_pool 2026-02-17T06:15:03.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=POBJ 2026-02-17T06:15:03.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map psr_pool POBJ 2026-02-17T06:15:03.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:15:03.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5777: TEST_periodic_scrub_replicated: local primary=0 2026-02-17T06:15:03.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5778: TEST_periodic_scrub_replicated: get_pg psr_pool POBJ 2026-02-17T06:15:03.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=psr_pool 2026-02-17T06:15:03.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=POBJ 2026-02-17T06:15:03.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map psr_pool POBJ 2026-02-17T06:15:03.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5778: TEST_periodic_scrub_replicated: local pg=2.0 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5781: TEST_periodic_scrub_replicated: local payload=UVWXYZ 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5782: TEST_periodic_scrub_replicated: echo UVWXYZ 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5784: TEST_periodic_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:15:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:15:03.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:15:03.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:15:03.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:15:03.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:15:03.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:15:03.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:15:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:15:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:15:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:15:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:15:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:15:04.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:15:04.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:15:04.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:15:05.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:15:05.804 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:05.805 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:05.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:05.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:05.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:15:05.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:15:05.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:15:05.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:15:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:15:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:15:05.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:15:05.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:15:05.829 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:05.821+0000 7f80226af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:05.830 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:05.821+0000 7f80226af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:05.831 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:05.825+0000 7f80226af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:05.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:15:05.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:15:05.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:15:05.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:15:05.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:15:05.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:05.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:15:05.962 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:15:05.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:05.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:06.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:07.065 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:07.057+0000 7f80226af8c0 -1 Falling back to public interface 2026-02-17T06:15:07.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:07.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:07.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:15:07.114 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:15:07.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:07.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:07.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:08.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:08.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:08.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:15:08.263 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:15:08.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:08.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:08.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:09.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:09.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:09.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:15:09.413 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:15:09.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:09.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:09.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:10.093 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:10.085+0000 7f80226af8c0 -1 osd.0 22 log_to_monitors true 2026-02-17T06:15:10.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:10.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:10.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:15:10.566 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:15:10.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:10.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 26 up_thru 26 down_at 23 last_clean_interval [5,22) [v2:127.0.0.1:6802/3905253949,v1:127.0.0.1:6803/3905253949] [v2:127.0.0.1:6804/3905253949,v1:127.0.0.1:6805/3905253949] exists,up e021134a-7e18-45c4-a735-8a1535137cd3 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:15:10.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:15:10.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:15:10.711 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:15:10.711 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:15:10.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:15:10.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:15:10.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:15:10.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:15:10.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:15:10.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:15:10.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:15:10.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:15:10.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:15:10.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:15:10.979 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:15:10.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:15:10.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:10.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:15:11.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149698 2026-02-17T06:15:11.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149698 2026-02-17T06:15:11.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149698' 2026-02-17T06:15:11.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:11.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:15:11.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-17T06:15:11.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-17T06:15:11.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149698 1-38654705671' 2026-02-17T06:15:11.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:11.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149698 2026-02-17T06:15:11.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:11.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:15:11.124 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:11.124 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149698 2026-02-17T06:15:11.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149698 2026-02-17T06:15:11.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149698' 2026-02-17T06:15:11.125 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 111669149698 2026-02-17T06:15:11.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:11.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 111669149698 2026-02-17T06:15:11.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:12.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:15:12.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:12.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 111669149698 2026-02-17T06:15:12.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:13.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:15:13.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:13.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149698 -lt 111669149698 2026-02-17T06:15:13.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:13.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-17T06:15:13.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:13.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:15:13.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-17T06:15:13.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:13.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-17T06:15:13.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T06:15:13.596 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705671 2026-02-17T06:15:13.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:15:13.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T06:15:13.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:15:13.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:15:13.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:15:13.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T06:15:13.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:15:13.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:15:13.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:15:13.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:15:13.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:15:13.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:15:13.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:15:14.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T06:15:14.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:15:14.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:15:14.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:15:14.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T06:15:14.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:15:14.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:15:14.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5787: TEST_periodic_scrub_replicated: set -o pipefail 2026-02-17T06:15:14.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5788: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T06:15:14.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5788: TEST_periodic_scrub_replicated: jq . 2026-02-17T06:15:14.289 INFO:tasks.workunit.client.0.trial074.stderr:No scrub information available for pg 2.0 2026-02-17T06:15:14.289 INFO:tasks.workunit.client.0.trial074.stderr:error 2: (2) No such file or directory 2026-02-17T06:15:14.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5789: TEST_periodic_scrub_replicated: set +o pipefail 2026-02-17T06:15:14.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5791: TEST_periodic_scrub_replicated: pg_deep_scrub 2.0 2026-02-17T06:15:14.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=2.0 2026-02-17T06:15:14.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 2.0 2026-02-17T06:15:14.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=2.0 2026-02-17T06:15:14.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:15:14.293 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:15:14.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:15:14.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:15:14.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:15:14.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:15:14.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:15:14.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:15:14.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:15:14.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:15:14.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:15:14.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:15:14.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:15:14.746 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:15:14.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:15:14.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:14.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:15:14.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149699 2026-02-17T06:15:14.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149699 2026-02-17T06:15:14.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699' 2026-02-17T06:15:14.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:14.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:15:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705673 2026-02-17T06:15:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705673 2026-02-17T06:15:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699 1-38654705673' 2026-02-17T06:15:14.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:14.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149699 2026-02-17T06:15:14.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:14.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:15:14.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149699 2026-02-17T06:15:14.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:14.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149699 2026-02-17T06:15:14.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149699' 2026-02-17T06:15:14.894 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 111669149699 2026-02-17T06:15:14.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:15.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149698 -lt 111669149699 2026-02-17T06:15:15.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:16.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:15:16.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-17T06:15:16.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:16.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705673 2026-02-17T06:15:16.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:16.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:15:16.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705673 2026-02-17T06:15:16.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:16.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705673 2026-02-17T06:15:16.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-17T06:15:16.213 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705673 2026-02-17T06:15:16.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:15:16.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705673 -lt 38654705673 2026-02-17T06:15:16.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:15:16.365 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 2.0 loop 0 2026-02-17T06:15:16.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 2.0 loop 0' 2026-02-17T06:15:16.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 2.0 2026-02-17T06:15:16.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=2.0 2026-02-17T06:15:16.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:15:16.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 2.0 query 2026-02-17T06:15:16.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:15:16.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:15:16.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:15:16.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:15:16.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:15:16.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T06:15:16.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:16.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:15:16.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:16.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T06:15:16.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T06:14:57.970385+0000 2026-02-17T06:15:16.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 2.0 2026-02-17T06:15:16.727 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0 on osd.0 to deep-scrub 2026-02-17T06:15:16.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 2.0 2026-02-17T06:14:57.970385+0000 last_deep_scrub_stamp 2026-02-17T06:15:16.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:14:57.970385+0000 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:16.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T06:15:16.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:14:57.970385+0000 '>' 2026-02-17T06:14:57.970385+0000 2026-02-17T06:15:16.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:17.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:17.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:17.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T06:15:17.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:17.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:15:17.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:17.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T06:15:18.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:14:57.970385+0000 '>' 2026-02-17T06:14:57.970385+0000 2026-02-17T06:15:18.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:19.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:19.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:19.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T06:15:19.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:19.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:15:19.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:19.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T06:15:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:14:57.970385+0000 '>' 2026-02-17T06:14:57.970385+0000 2026-02-17T06:15:19.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:20.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:20.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:20.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T06:15:20.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:20.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:15:20.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:20.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T06:15:20.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:14:57.970385+0000 '>' 2026-02-17T06:14:57.970385+0000 2026-02-17T06:15:20.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:21.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:21.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:21.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T06:15:21.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:21.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:15:21.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:21.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T06:15:21.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:15:17.079439+0000 '>' 2026-02-17T06:14:57.970385+0000 2026-02-17T06:15:21.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:15:21.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5794: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T06:15:21.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5794: TEST_periodic_scrub_replicated: jq . 2026-02-17T06:15:21.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5794: TEST_periodic_scrub_replicated: grep -q POBJ 2026-02-17T06:15:21.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5796: TEST_periodic_scrub_replicated: flush_pg_stats 2026-02-17T06:15:21.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:15:21.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:15:21.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:15:21.674 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:15:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:15:21.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:21.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:15:21.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149702 2026-02-17T06:15:21.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149702 2026-02-17T06:15:21.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149702' 2026-02-17T06:15:21.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:21.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:15:21.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T06:15:21.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T06:15:21.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149702 1-38654705675' 2026-02-17T06:15:21.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:21.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149702 2026-02-17T06:15:21.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:21.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:15:21.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149702 2026-02-17T06:15:21.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:21.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149702 2026-02-17T06:15:21.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149702' 2026-02-17T06:15:21.821 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 111669149702 2026-02-17T06:15:21.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:21.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149701 -lt 111669149702 2026-02-17T06:15:21.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:22.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:15:22.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:23.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-17T06:15:23.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:23.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T06:15:23.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:23.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:15:23.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T06:15:23.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:23.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T06:15:23.132 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705675 2026-02-17T06:15:23.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T06:15:23.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:15:23.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T06:15:23.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5797: TEST_periodic_scrub_replicated: get_last_scrub_stamp 2.0 2026-02-17T06:15:23.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:23.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:23.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:23.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:23.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5797: TEST_periodic_scrub_replicated: local last_scrub=2026-02-17T06:15:17.079439+0000 2026-02-17T06:15:23.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5799: TEST_periodic_scrub_replicated: ceph tell 2.0 schedule-scrub 2026-02-17T06:15:23.507 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:15:23.507 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:15:23.507 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:15:23.508 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:13:43.504807+0000" 2026-02-17T06:15:23.508 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:15:23.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5801: TEST_periodic_scrub_replicated: wait_for_scrub 2.0 2026-02-17T06:15:17.079439+0000 2026-02-17T06:15:23.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T06:15:23.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:15:17.079439+0000 2026-02-17T06:15:23.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:15:23.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:15:23.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:23.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:23.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:23.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:23.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:23.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:23.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:15:17.079439+0000 '>' 2026-02-17T06:15:17.079439+0000 2026-02-17T06:15:23.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:24.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:24.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:24.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:24.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:24.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:24.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:24.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:24.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:15:17.079439+0000 '>' 2026-02-17T06:15:17.079439+0000 2026-02-17T06:15:24.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:25.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:25.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:25.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:25.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:25.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:25.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:25.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:25.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:15:17.079439+0000 '>' 2026-02-17T06:15:17.079439+0000 2026-02-17T06:15:25.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:26.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:26.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:26.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:26.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:26.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:26.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:26.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:27.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:15:24.040533+0000 '>' 2026-02-17T06:15:17.079439+0000 2026-02-17T06:15:27.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:15:27.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5804: TEST_periodic_scrub_replicated: grep -q 'Deep scrub errors, upgrading scrub to deep-scrub' td/osd-scrub-repair/osd.0.log 2026-02-17T06:15:27.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5807: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T06:15:27.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5807: TEST_periodic_scrub_replicated: jq . 2026-02-17T06:15:27.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5807: TEST_periodic_scrub_replicated: grep -q POBJ 2026-02-17T06:15:27.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5810: TEST_periodic_scrub_replicated: ceph osd set nodeep-scrub 2026-02-17T06:15:27.349 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:15:27.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5812: TEST_periodic_scrub_replicated: ceph tell osd.0 get_latest_osdmap 2026-02-17T06:15:27.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5813: TEST_periodic_scrub_replicated: flush_pg_stats 2026-02-17T06:15:27.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:15:27.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:15:27.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:15:27.597 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:15:27.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:15:27.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:27.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:15:27.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149704 2026-02-17T06:15:27.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149704 2026-02-17T06:15:27.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149704' 2026-02-17T06:15:27.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:27.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:15:27.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705677 2026-02-17T06:15:27.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705677 2026-02-17T06:15:27.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149704 1-38654705677' 2026-02-17T06:15:27.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:27.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149704 2026-02-17T06:15:27.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:27.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:15:27.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149704 2026-02-17T06:15:27.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:27.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149704 2026-02-17T06:15:27.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149704' 2026-02-17T06:15:27.746 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 111669149704 2026-02-17T06:15:27.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:27.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149703 -lt 111669149704 2026-02-17T06:15:27.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:28.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:15:28.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:29.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149703 -lt 111669149704 2026-02-17T06:15:29.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:30.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:15:30.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:30.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149704 -lt 111669149704 2026-02-17T06:15:30.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:30.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705677 2026-02-17T06:15:30.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:30.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:15:30.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705677 2026-02-17T06:15:30.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:30.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705677 2026-02-17T06:15:30.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-17T06:15:30.214 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705677 2026-02-17T06:15:30.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:15:30.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705677 -lt 38654705677 2026-02-17T06:15:30.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5814: TEST_periodic_scrub_replicated: sleep 5 2026-02-17T06:15:35.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5817: TEST_periodic_scrub_replicated: ceph tell 2.0 schedule-scrub 2026-02-17T06:15:35.446 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:15:35.446 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:15:35.446 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:15:35.446 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:13:55.443329+0000" 2026-02-17T06:15:35.446 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:15:35.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5820: TEST_periodic_scrub_replicated: local found=false 2026-02-17T06:15:35.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5821: TEST_periodic_scrub_replicated: seq 14 -1 0 2026-02-17T06:15:35.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5821: TEST_periodic_scrub_replicated: for i in $(seq 14 -1 0) 2026-02-17T06:15:35.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5823: TEST_periodic_scrub_replicated: sleep 1 2026-02-17T06:15:36.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5824: TEST_periodic_scrub_replicated: grep -q 'Regular scrub skipped due to deep-scrub errors and nodeep-scrub set' td/osd-scrub-repair/osd.0.log 2026-02-17T06:15:36.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5824: TEST_periodic_scrub_replicated: found=true 2026-02-17T06:15:36.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5824: TEST_periodic_scrub_replicated: break 2026-02-17T06:15:36.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5827: TEST_periodic_scrub_replicated: test true = true 2026-02-17T06:15:36.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5830: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T06:15:36.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5830: TEST_periodic_scrub_replicated: jq . 2026-02-17T06:15:36.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5830: TEST_periodic_scrub_replicated: grep -q POBJ 2026-02-17T06:15:36.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5832: TEST_periodic_scrub_replicated: flush_pg_stats 2026-02-17T06:15:36.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:15:36.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:15:36.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:15:36.643 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:15:36.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:15:36.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:36.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:15:36.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149707 2026-02-17T06:15:36.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149707 2026-02-17T06:15:36.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149707' 2026-02-17T06:15:36.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:36.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:15:36.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705680 2026-02-17T06:15:36.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705680 2026-02-17T06:15:36.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149707 1-38654705680' 2026-02-17T06:15:36.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:36.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149707 2026-02-17T06:15:36.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:36.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:15:36.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149707 2026-02-17T06:15:36.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:36.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149707 2026-02-17T06:15:36.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149707' 2026-02-17T06:15:36.791 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 111669149707 2026-02-17T06:15:36.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:36.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149705 -lt 111669149707 2026-02-17T06:15:36.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:37.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:15:37.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:38.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149707 -lt 111669149707 2026-02-17T06:15:38.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:38.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705680 2026-02-17T06:15:38.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:38.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:15:38.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705680 2026-02-17T06:15:38.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:38.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705680 2026-02-17T06:15:38.100 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705680 2026-02-17T06:15:38.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-17T06:15:38.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:15:38.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-17T06:15:38.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5834: TEST_periodic_scrub_replicated: pg_scrub 2.0 2026-02-17T06:15:38.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=2.0 2026-02-17T06:15:38.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 2.0 2026-02-17T06:15:38.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=2.0 2026-02-17T06:15:38.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:15:38.252 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:15:38.252 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:15:38.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:15:38.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:15:38.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:15:38.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:15:38.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:15:38.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:15:38.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:15:38.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:15:38.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:15:38.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:15:38.695 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:15:38.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:15:38.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:38.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:15:38.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149708 2026-02-17T06:15:38.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149708 2026-02-17T06:15:38.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149708' 2026-02-17T06:15:38.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:15:38.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:15:38.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705681 2026-02-17T06:15:38.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705681 2026-02-17T06:15:38.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149708 1-38654705681' 2026-02-17T06:15:38.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:38.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149708 2026-02-17T06:15:38.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:38.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:15:38.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149708 2026-02-17T06:15:38.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:38.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149708 2026-02-17T06:15:38.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149708' 2026-02-17T06:15:38.842 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 111669149708 2026-02-17T06:15:38.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:39.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149707 -lt 111669149708 2026-02-17T06:15:39.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:15:40.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:15:40.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:15:40.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149708 -lt 111669149708 2026-02-17T06:15:40.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:15:40.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705681 2026-02-17T06:15:40.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:15:40.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:15:40.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705681 2026-02-17T06:15:40.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:15:40.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705681 2026-02-17T06:15:40.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705681' 2026-02-17T06:15:40.160 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705681 2026-02-17T06:15:40.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:15:40.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705681 -lt 38654705681 2026-02-17T06:15:40.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:15:40.320 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 2.0 loop 0 2026-02-17T06:15:40.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 2.0 loop 0' 2026-02-17T06:15:40.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 2.0 2026-02-17T06:15:40.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=2.0 2026-02-17T06:15:40.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:15:40.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 2.0 query 2026-02-17T06:15:40.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:15:40.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-17T06:15:40.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:15:40.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:15:40.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:15:40.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 2.0 2026-02-17T06:15:40.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:40.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:40.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:40.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:40.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:40.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 2.0 2026-02-17T06:15:40.684 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 2.0 on osd.0 to scrub 2026-02-17T06:15:40.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 2.0 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:40.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T06:15:40.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:40.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:15:40.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:15:40.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:40.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:40.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:40.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:40.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:40.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:40.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:13:55.443329+0000 '>' 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:40.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:41.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:41.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:41.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:41.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:41.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:41.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:41.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:41.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:13:55.443329+0000 '>' 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:41.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:42.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:42.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:42.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:42.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:42.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:42.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:42.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:43.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:13:55.443329+0000 '>' 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:43.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:44.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:44.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:44.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:44.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:44.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:44.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:44.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:44.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:13:55.443329+0000 '>' 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:44.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:45.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:45.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:45.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:45.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:45.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:45.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:45.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:45.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:13:55.443329+0000 '>' 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:45.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:46.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:46.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:46.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:46.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:46.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:46.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:46.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:46.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:13:55.443329+0000 '>' 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:46.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:15:47.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:15:47.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:15:47.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T06:15:47.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T06:15:47.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:15:47.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:15:47.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T06:15:47.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:15:40.992801+0000 '>' 2026-02-10T06:13:55.443329+0000 2026-02-17T06:15:47.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:15:47.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5835: TEST_periodic_scrub_replicated: grep -q 'Regular scrub request, deep-scrub details will be lost' td/osd-scrub-repair/osd.0.log 2026-02-17T06:15:47.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5838: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T06:15:47.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5838: TEST_periodic_scrub_replicated: jq . 2026-02-17T06:15:47.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5838: TEST_periodic_scrub_replicated: grep -qv POBJ 2026-02-17T06:15:47.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:15:47.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:15:47.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:15:47.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:15:47.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:15:47.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:15:47.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:15:47.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:15:47.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:15:47.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:15:47.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:15:47.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:15:47.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:15:47.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:15:47.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:15:47.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:15:47.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:15:47.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:15:47.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:15:47.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:15:47.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:15:47.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:15:47.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:15:47.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:15:47.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:47.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:47.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:15:47.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:15:47.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:15:47.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:15:47.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:15:47.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:15:47.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:15:47.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:15:47.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:15:47.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:15:47.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:15:47.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:15:47.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:15:47.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:15:47.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:15:47.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:15:47.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:15:47.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:15:47.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:15:47.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:15:47.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:47.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:47.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:15:47.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:15:47.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:15:47.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:15:47.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:15:47.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:47.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:47.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:15:47.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:15:47.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:15:47.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_repair_stats td/osd-scrub-repair 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:654: TEST_repair_stats: local dir=td/osd-scrub-repair 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:655: TEST_repair_stats: local poolname=testpool 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:656: TEST_repair_stats: local OSDS=2 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:657: TEST_repair_stats: local OBJS=30 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:659: TEST_repair_stats: local REPAIRS=20 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:662: TEST_repair_stats: run_mon td/osd-scrub-repair a 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:15:47.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:15:47.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:15:47.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:15:47.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:15:47.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:15:47.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:15:47.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:15:47.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:15:47.954 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:15:47.954 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:47.954 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:47.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:47.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:15:47.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:15:47.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:15:47.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:15:47.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:15:47.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:15:47.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:15:47.984 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:15:47.985 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:15:47.985 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:15:47.985 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:15:47.985 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:47.985 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:47.985 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:15:47.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:15:47.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:15:48.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:15:48.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:15:48.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:15:48.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:15:48.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:15:48.050 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:15:48.050 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:15:48.051 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:15:48.052 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:15:48.052 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:48.052 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:48.052 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:15:48.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:15:48.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:15:48.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:663: TEST_repair_stats: run_mgr td/osd-scrub-repair x 2026-02-17T06:15:48.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:15:48.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:15:48.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:15:48.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:15:48.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:15:48.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:15:48.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:15:48.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:15:48.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:15:48.208 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:15:48.208 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:48.208 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:48.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:48.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:15:48.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:15:48.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:665: TEST_repair_stats: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:15:48.227 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: expr 2 - 1 2026-02-17T06:15:48.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: seq 0 1 2026-02-17T06:15:48.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:15:48.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:667: TEST_repair_stats: run_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:15:48.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:15:48.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:15:48.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:15:48.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:15:48.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:15:48.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:15:48.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:15:48.234 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:15:48.234 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:48.234 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:48.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:48.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:48.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:15:48.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:15:48.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:15:48.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:15:48.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:15:48.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:15:48.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:15:48.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:15:48.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:15:48.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:15:48.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:15:48.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:15:48.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:15:48.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ee8de83b-cbac-4bfa-ae45-0ad185e8a51b 2026-02-17T06:15:48.237 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 ee8de83b-cbac-4bfa-ae45-0ad185e8a51b 2026-02-17T06:15:48.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 ee8de83b-cbac-4bfa-ae45-0ad185e8a51b' 2026-02-17T06:15:48.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:15:48.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCUB5RpW+VaDhAAR6n5JFEIAJbJy+CRKgH+IQ== 2026-02-17T06:15:48.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCUB5RpW+VaDhAAR6n5JFEIAJbJy+CRKgH+IQ=="}' 2026-02-17T06:15:48.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ee8de83b-cbac-4bfa-ae45-0ad185e8a51b -i td/osd-scrub-repair/0/new.json 2026-02-17T06:15:48.334 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:15:48.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:15:48.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQCUB5RpW+VaDhAAR6n5JFEIAJbJy+CRKgH+IQ== --osd-uuid ee8de83b-cbac-4bfa-ae45-0ad185e8a51b 2026-02-17T06:15:48.358 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:48.349+0000 7f32cb2638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:48.359 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:48.353+0000 7f32cb2638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:48.360 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:48.353+0000 7f32cb2638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:48.361 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:48.353+0000 7f32cb2638c0 -1 bdev(0x55f8567cfc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:15:48.361 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:48.353+0000 7f32cb2638c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:15:50.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:15:50.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:15:50.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:15:50.986 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:15:50.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:15:51.183 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:15:51.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:15:51.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:15:51.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:15:51.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:15:51.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:15:51.200 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:51.193+0000 7f0a689228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:51.201 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:51.193+0000 7f0a689228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:51.202 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:51.193+0000 7f0a689228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:51.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:15:51.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:15:51.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:15:51.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:15:51.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:15:51.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:51.336 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:15:51.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:15:51.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:51.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:51.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:52.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:52.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:52.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:15:52.490 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:15:52.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:52.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:52.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:52.717 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:52.709+0000 7f0a689228c0 -1 Falling back to public interface 2026-02-17T06:15:53.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:53.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:53.644 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:15:53.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:15:53.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:53.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:53.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:54.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:54.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:54.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:15:54.796 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:15:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:54.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:54.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:55.735 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:55.725+0000 7f0a689228c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:15:55.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:55.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:55.949 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:15:55.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:15:55.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:55.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:56.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:15:57.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:15:57.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:15:57.105 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:15:57.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:15:57.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:15:57.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:15:57.257 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1469973623,v1:127.0.0.1:6803/1469973623] [v2:127.0.0.1:6804/1469973623,v1:127.0.0.1:6805/1469973623] exists,up ee8de83b-cbac-4bfa-ae45-0ad185e8a51b 2026-02-17T06:15:57.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:15:57.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:15:57.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:15:57.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:15:57.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:667: TEST_repair_stats: run_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:15:57.258 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:15:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:15:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:15:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:15:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:15:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:15:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:15:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:15:57.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:15:57.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=41923490-d846-4fd9-b3d0-04842b7b4002 2026-02-17T06:15:57.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 41923490-d846-4fd9-b3d0-04842b7b4002' 2026-02-17T06:15:57.263 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 41923490-d846-4fd9-b3d0-04842b7b4002 2026-02-17T06:15:57.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:15:57.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCdB5RpXjIrEBAAgx5k3WE0g4D8kTLcoy/7Vg== 2026-02-17T06:15:57.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCdB5RpXjIrEBAAgx5k3WE0g4D8kTLcoy/7Vg=="}' 2026-02-17T06:15:57.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 41923490-d846-4fd9-b3d0-04842b7b4002 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:15:57.431 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:15:57.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:15:57.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQCdB5RpXjIrEBAAgx5k3WE0g4D8kTLcoy/7Vg== --osd-uuid 41923490-d846-4fd9-b3d0-04842b7b4002 2026-02-17T06:15:57.458 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:57.449+0000 7fa5c60cb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:57.459 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:57.453+0000 7fa5c60cb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:57.460 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:57.453+0000 7fa5c60cb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:57.461 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:57.453+0000 7fa5c60cb8c0 -1 bdev(0x55dd12797c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:15:57.461 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:57.453+0000 7fa5c60cb8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:15:59.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:15:59.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:15:59.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:15:59.769 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:15:59.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:15:59.965 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:15:59.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:15:59.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:15:59.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:15:59.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:15:59.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:15:59.989 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:59.981+0000 7f9a3e4288c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:59.991 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:59.981+0000 7f9a3e4288c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:15:59.992 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:15:59.985+0000 7f9a3e4288c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:16:00.117 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:16:00.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:00.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:00.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:01.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:01.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:01.271 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:16:01.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:16:01.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:01.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:01.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:01.477 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:01.469+0000 7f9a3e4288c0 -1 Falling back to public interface 2026-02-17T06:16:02.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:02.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:02.425 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:16:02.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:16:02.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:02.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:02.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:03.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:03.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:03.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:16:03.578 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:16:03.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:03.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:03.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:04.466 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:04.457+0000 7f9a3e4288c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:16:04.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:04.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:04.731 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:16:04.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:16:04.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:04.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:04.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:05.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:05.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:05.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:16:05.884 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:16:05.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:05.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:06.033 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1860435949,v1:127.0.0.1:6811/1860435949] [v2:127.0.0.1:6812/1860435949,v1:127.0.0.1:6813/1860435949] exists,up 41923490-d846-4fd9-b3d0-04842b7b4002 2026-02-17T06:16:06.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:16:06.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:16:06.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:16:06.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:670: TEST_repair_stats: create_pool testpool 1 1 2026-02-17T06:16:06.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T06:16:06.233 INFO:tasks.workunit.client.0.trial074.stderr:pool 'testpool' created 2026-02-17T06:16:06.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:16:07.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:671: TEST_repair_stats: ceph osd pool set testpool size 2 2026-02-17T06:16:07.437 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 size to 2 2026-02-17T06:16:07.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:672: TEST_repair_stats: wait_for_clean 2026-02-17T06:16:07.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:16:07.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:16:07.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:16:07.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:16:07.448 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:16:07.448 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:16:07.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:16:07.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:16:07.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:16:07.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:16:07.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:16:07.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:16:07.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:16:07.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:16:07.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:16:07.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:16:07.728 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:16:07.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:16:07.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:16:07.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:16:07.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:16:07.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:16:07.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:16:07.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:16:07.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:16:07.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T06:16:07.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T06:16:07.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T06:16:07.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:16:07.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:16:07.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:16:07.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:16:07.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:16:07.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:16:07.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:16:07.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:16:07.878 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:16:07.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:16:08.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T06:16:08.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:16:09.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:16:09.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:16:09.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:16:09.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:16:09.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T06:16:09.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:16:09.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:16:09.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T06:16:09.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:16:09.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T06:16:09.195 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672962 2026-02-17T06:16:09.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T06:16:09.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:16:09.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T06:16:09.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:16:09.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:09.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:09.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:16:09.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:16:09.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:16:09.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:16:09.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:16:09.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:16:09.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:16:09.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:16:09.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:16:09.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:16:09.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:09.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:09.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:16:09.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:16:09.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:16:09.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:675: TEST_repair_stats: local payload=ABCDEF 2026-02-17T06:16:09.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:676: TEST_repair_stats: echo ABCDEF 2026-02-17T06:16:09.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: seq 1 30 2026-02-17T06:16:09.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:09.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:09.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:09.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:09.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:09.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:09.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:09.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:09.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:09.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:09.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:09.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:09.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:09.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj11 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj12 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj13 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj14 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj15 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj16 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj17 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj18 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj19 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj20 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj21 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj22 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj23 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj24 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj25 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj26 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj27 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj28 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj29 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T06:16:10.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj30 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:16:10.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:684: TEST_repair_stats: get_not_primary testpool obj1 2026-02-17T06:16:10.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T06:16:10.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T06:16:10.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool obj1 2026-02-17T06:16:10.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T06:16:10.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:16:10.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T06:16:10.592 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:16:10.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T06:16:10.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool obj1 2026-02-17T06:16:10.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T06:16:10.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:684: TEST_repair_stats: local other=0 2026-02-17T06:16:10.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_repair_stats: get_pg testpool obj1 2026-02-17T06:16:10.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T06:16:10.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T06:16:10.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T06:16:10.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:16:11.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_repair_stats: local pgid=1.0 2026-02-17T06:16:11.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:686: TEST_repair_stats: get_primary testpool obj1 2026-02-17T06:16:11.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T06:16:11.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:16:11.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:16:11.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T06:16:11.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:686: TEST_repair_stats: local primary=1 2026-02-17T06:16:11.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:688: TEST_repair_stats: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:16:11.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:16:11.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:16:11.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:16:11.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:16:11.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:16:12.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:16:12.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:689: TEST_repair_stats: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:16:12.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:16:12.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:16:12.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:16:12.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:16:12.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:16:13.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:16:13.831 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: seq 1 20 2026-02-17T06:16:13.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:13.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 1 % 2 2026-02-17T06:16:13.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:13.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 remove 2026-02-17T06:16:13.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:13.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:13.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:13.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:13.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:13.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 remove 2026-02-17T06:16:14.491 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T06:16:15.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:15.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 2 % 2 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:15.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T06:16:15.668 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T06:16:16.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:16.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 3 % 2 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj3 remove 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:16.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj3 remove 2026-02-17T06:16:16.857 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:8dd16f86:::obj3:head# 2026-02-17T06:16:17.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:17.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 4 % 2 2026-02-17T06:16:17.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:17.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj4 remove 2026-02-17T06:16:17.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:17.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:17.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:17.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:17.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:17.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj4 remove 2026-02-17T06:16:18.030 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:0ee9ae15:::obj4:head# 2026-02-17T06:16:18.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:18.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 5 % 2 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj5 remove 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:18.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj5 remove 2026-02-17T06:16:19.192 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:c52c9666:::obj5:head# 2026-02-17T06:16:19.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:19.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 6 % 2 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj6 remove 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:19.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj6 remove 2026-02-17T06:16:20.380 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:233a42c1:::obj6:head# 2026-02-17T06:16:20.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:20.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 7 % 2 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj7 remove 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj7 remove 2026-02-17T06:16:21.552 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:9a09113a:::obj7:head# 2026-02-17T06:16:22.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:22.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 8 % 2 2026-02-17T06:16:22.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:22.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj8 remove 2026-02-17T06:16:22.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:22.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:22.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:22.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:22.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:22.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj8 remove 2026-02-17T06:16:22.732 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:a557efb1:::obj8:head# 2026-02-17T06:16:23.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:23.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 9 % 2 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj9 remove 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:23.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj9 remove 2026-02-17T06:16:23.901 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:fe351e27:::obj9:head# 2026-02-17T06:16:24.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:24.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 10 % 2 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj10 remove 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:24.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj10 remove 2026-02-17T06:16:25.064 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:d1337354:::obj10:head# 2026-02-17T06:16:25.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:25.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 11 % 2 2026-02-17T06:16:25.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:25.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj11 remove 2026-02-17T06:16:25.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:25.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:25.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:25.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:25.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:25.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj11 remove 2026-02-17T06:16:26.259 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:eeae2a94:::obj11:head# 2026-02-17T06:16:26.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:26.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 12 % 2 2026-02-17T06:16:26.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:26.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj12 remove 2026-02-17T06:16:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj12 remove 2026-02-17T06:16:27.428 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:af5f95cb:::obj12:head# 2026-02-17T06:16:27.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:27.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 13 % 2 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj13 remove 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:27.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj13 remove 2026-02-17T06:16:28.593 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:a61ad63e:::obj13:head# 2026-02-17T06:16:29.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:29.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 14 % 2 2026-02-17T06:16:29.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:29.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj14 remove 2026-02-17T06:16:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj14 remove 2026-02-17T06:16:29.764 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:e0a44829:::obj14:head# 2026-02-17T06:16:30.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:30.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 15 % 2 2026-02-17T06:16:30.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:30.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj15 remove 2026-02-17T06:16:30.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:30.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:30.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:30.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:30.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:30.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj15 remove 2026-02-17T06:16:30.939 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:ab946124:::obj15:head# 2026-02-17T06:16:31.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:31.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 16 % 2 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj16 remove 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:31.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj16 remove 2026-02-17T06:16:32.124 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:461f8b5e:::obj16:head# 2026-02-17T06:16:32.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:32.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 17 % 2 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj17 remove 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj17 remove 2026-02-17T06:16:33.304 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:24fc6e92:::obj17:head# 2026-02-17T06:16:33.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:33.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 18 % 2 2026-02-17T06:16:33.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:33.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj18 remove 2026-02-17T06:16:33.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:33.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:33.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:33.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:33.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:33.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj18 remove 2026-02-17T06:16:34.471 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:b9836a99:::obj18:head# 2026-02-17T06:16:35.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:35.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 19 % 2 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj19 remove 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj19 remove 2026-02-17T06:16:35.664 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:00b31b6c:::obj19:head# 2026-02-17T06:16:36.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T06:16:36.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 20 % 2 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj20 remove 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj20 remove 2026-02-17T06:16:36.844 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:24b5611b:::obj20:head# 2026-02-17T06:16:37.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:696: TEST_repair_stats: activate_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:16:37.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:16:37.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:16:37.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:16:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:16:37.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:16:37.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:16:37.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:16:37.389 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:16:37.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:16:37.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:16:37.391 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:16:37.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:16:37.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:16:37.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:16:37.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:16:37.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:16:37.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:16:37.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:16:37.413 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:37.405+0000 7f8e347b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:16:37.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:37.405+0000 7f8e347b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:16:37.415 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:37.409+0000 7f8e347b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:16:37.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:16:37.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:16:37.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:16:37.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:16:37.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:16:37.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:37.544 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:16:37.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:16:37.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:37.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:37.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:38.653 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:38.645+0000 7f8e347b28c0 -1 Falling back to public interface 2026-02-17T06:16:38.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:38.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:38.697 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:16:38.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:16:38.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:38.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:38.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:39.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:39.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:39.845 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:16:39.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:16:39.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:39.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:39.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:40.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:40.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:40.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:16:40.997 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:16:40.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:40.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:41.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:41.652 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:41.648+0000 7f8e347b28c0 -1 osd.1 18 log_to_monitors true 2026-02-17T06:16:42.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:42.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:42.148 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:16:42.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:16:42.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:42.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:16:42.298 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 20 up_thru 20 down_at 18 last_clean_interval [10,18) [v2:127.0.0.1:6802/2026876541,v1:127.0.0.1:6803/2026876541] [v2:127.0.0.1:6804/2026876541,v1:127.0.0.1:6805/2026876541] exists,up 41923490-d846-4fd9-b3d0-04842b7b4002 2026-02-17T06:16:42.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:697: TEST_repair_stats: activate_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:16:42.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:16:42.303 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:16:42.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:16:42.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:16:42.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:16:42.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:16:42.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:16:42.305 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:16:42.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:16:42.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:16:42.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:16:42.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:16:42.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:16:42.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:16:42.320 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:42.316+0000 7f56a3b518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:16:42.321 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:42.316+0000 7f56a3b518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:16:42.322 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:42.316+0000 7f56a3b518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:16:42.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:16:42.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:16:42.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:16:42.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:16:42.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:16:42.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:42.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:16:42.461 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:16:42.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:42.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:16:42.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:43.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:43.560+0000 7f56a3b518c0 -1 Falling back to public interface 2026-02-17T06:16:43.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:43.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:43.611 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:16:43.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:16:43.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:43.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:16:43.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:44.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:16:44.761 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:16:44.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:44.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:16:44.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:45.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:45.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:45.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:16:45.911 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:16:45.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:45.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:16:46.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:16:46.543 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:16:46.540+0000 7f56a3b518c0 -1 osd.0 15 log_to_monitors true 2026-02-17T06:16:47.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:16:47.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:16:47.061 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:16:47.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:16:47.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:16:47.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:16:47.207 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 24 up_thru 0 down_at 16 last_clean_interval [5,15) [v2:127.0.0.1:6810/2939900092,v1:127.0.0.1:6811/2939900092] [v2:127.0.0.1:6812/2939900092,v1:127.0.0.1:6813/2939900092] exists,up ee8de83b-cbac-4bfa-ae45-0ad185e8a51b 2026-02-17T06:16:47.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:16:47.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:16:47.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:16:47.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:698: TEST_repair_stats: wait_for_clean 2026-02-17T06:16:47.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:16:47.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:16:47.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:16:47.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:16:47.210 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:16:47.210 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:16:47.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:16:47.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:16:47.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:16:47.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:16:47.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:16:47.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:16:47.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:16:47.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:16:47.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:16:47.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:16:47.486 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:16:47.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:16:47.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:16:47.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:16:47.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215106 2026-02-17T06:16:47.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215106 2026-02-17T06:16:47.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106' 2026-02-17T06:16:47.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:16:47.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:16:47.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345923 2026-02-17T06:16:47.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345923 2026-02-17T06:16:47.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-85899345923' 2026-02-17T06:16:47.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:16:47.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215106 2026-02-17T06:16:47.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:16:47.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:16:47.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215106 2026-02-17T06:16:47.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:16:47.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215106 2026-02-17T06:16:47.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215106' 2026-02-17T06:16:47.635 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 103079215106 2026-02-17T06:16:47.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:16:47.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 103079215106 2026-02-17T06:16:47.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:16:48.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:16:48.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:16:48.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215106 -lt 103079215106 2026-02-17T06:16:48.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:16:48.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-85899345923 2026-02-17T06:16:48.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:16:48.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:16:48.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-85899345923 2026-02-17T06:16:48.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:16:48.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345923 2026-02-17T06:16:48.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 85899345923' 2026-02-17T06:16:48.956 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 85899345923 2026-02-17T06:16:48.956 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:16:49.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345923 -lt 85899345923 2026-02-17T06:16:49.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:16:49.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:49.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:49.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:16:49.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:16:49.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:16:49.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:16:49.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:16:49.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:16:49.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:16:49.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:16:49.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:16:49.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:16:49.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:49.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:49.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:16:49.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T06:16:49.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T06:16:49.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T06:16:49.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:16:49.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T06:16:49.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:16:49.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:16:49.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:16:49.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:16:49.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:16:49.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:16:49.739 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:16:49.739 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:16:49.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:16:49.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:16:49.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:49.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:16:50.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:16:50.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:16:50.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:16:50.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:16:50.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T06:16:50.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:16:50.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T06:16:50.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:16:50.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:16:50.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:16:50.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:16:50.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:16:50.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:16:50.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:16:50.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:16:50.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:16:50.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:16:50.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:50.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:16:50.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:16:50.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:16:50.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:16:50.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:16:50.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T06:16:50.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:16:50.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T06:16:51.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:16:51.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:16:51.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:16:51.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:16:51.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:16:51.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:16:51.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:16:51.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:16:51.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:16:51.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:16:51.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:51.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:51.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:16:51.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:16:51.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:16:51.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:16:51.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:16:51.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:16:51.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:16:51.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:16:51.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:16:51.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:16:51.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:16:51.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T06:16:51.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:16:51.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T06:16:52.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:16:52.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:16:52.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:16:52.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:16:52.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:16:52.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:16:52.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:16:52.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:16:52.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:16:52.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:16:52.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:16:52.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:16:53.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:16:53.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:700: TEST_repair_stats: repair 1.0 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:16:53.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:16:53.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:53.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T06:16:53.326 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T06:16:53.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:53.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:16:53.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:53.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:16:53.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:16:53.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:16:53.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:16:53.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:16:53.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:16:53.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:16:53.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:16:53.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:16:06.229627+0000 '>' 2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:53.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:16:54.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:16:54.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:16:54.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:16:54.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:16:54.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:16:54.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:16:54.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:16:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:16:06.229627+0000 '>' 2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:54.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:16:55.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:16:55.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:16:55.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:16:55.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:16:55.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:16:55.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:16:55.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:16:55.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:16:06.229627+0000 '>' 2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:55.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:16:56.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:16:56.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:16:56.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:16:56.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:16:56.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:16:56.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:16:56.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:16:56.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:16:06.229627+0000 '>' 2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:56.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:16:57.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:16:57.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:16:57.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:16:57.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:16:57.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:16:57.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:16:57.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:16:58.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:16:06.229627+0000 '>' 2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:58.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:16:59.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:16:59.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:16:59.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:16:59.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:16:59.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:16:59.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:16:59.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:16:59.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:16:53.567065+0000 '>' 2026-02-17T06:16:06.229627+0000 2026-02-17T06:16:59.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:16:59.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:701: TEST_repair_stats: wait_for_clean 2026-02-17T06:16:59.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:16:59.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:16:59.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:16:59.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:16:59.263 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:16:59.263 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:16:59.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:16:59.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:16:59.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:16:59.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:16:59.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:16:59.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:16:59.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:16:59.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:16:59.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:16:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:16:59.544 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:16:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:16:59.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:16:59.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:16:59.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215109 2026-02-17T06:16:59.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215109 2026-02-17T06:16:59.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215109' 2026-02-17T06:16:59.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:16:59.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:16:59.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345926 2026-02-17T06:16:59.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345926 2026-02-17T06:16:59.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215109 1-85899345926' 2026-02-17T06:16:59.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:16:59.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215109 2026-02-17T06:16:59.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:16:59.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:16:59.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215109 2026-02-17T06:16:59.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:16:59.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215109 2026-02-17T06:16:59.693 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 103079215109 2026-02-17T06:16:59.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215109' 2026-02-17T06:16:59.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:16:59.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215108 -lt 103079215109 2026-02-17T06:16:59.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:17:00.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:17:00.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:17:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215109 2026-02-17T06:17:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:17:01.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-85899345926 2026-02-17T06:17:01.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:17:01.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:17:01.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-85899345926 2026-02-17T06:17:01.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:17:01.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345926 2026-02-17T06:17:01.012 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 85899345926 2026-02-17T06:17:01.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 85899345926' 2026-02-17T06:17:01.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:17:01.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345926 -lt 85899345926 2026-02-17T06:17:01.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:17:01.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:17:01.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:17:01.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:17:01.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:17:01.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:17:01.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:17:01.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:17:01.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:17:01.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:17:01.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:17:01.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:17:01.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:17:01.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:17:01.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:17:01.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:17:01.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:17:01.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:17:01.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:702: TEST_repair_stats: ceph pg dump pgs 2026-02-17T06:17:01.820 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T06:17:01.820 INFO:tasks.workunit.client.0.trial074.stdout:1.0 30 0 0 0 0 210 0 0 30 0 30 active+clean 2026-02-17T06:16:53.583777+0000 15'30 25:129 [1,0] 1 [1,0] 1 15'30 2026-02-17T06:16:53.567065+0000 15'30 2026-02-17T06:16:53.567065+0000 0 1 periodic scrub scheduled @ 2026-02-18T06:16:53.567065+0000 20 0 2026-02-17T06:17:01.820 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T06:17:01.820 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T06:17:01.821 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:17:01.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:703: TEST_repair_stats: flush_pg_stats 2026-02-17T06:17:01.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:17:01.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:17:01.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:17:01.995 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:17:01.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:17:01.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:17:01.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:17:02.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215111 2026-02-17T06:17:02.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215111 2026-02-17T06:17:02.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215111' 2026-02-17T06:17:02.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:17:02.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:17:02.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345928 2026-02-17T06:17:02.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345928 2026-02-17T06:17:02.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215111 1-85899345928' 2026-02-17T06:17:02.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:17:02.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215111 2026-02-17T06:17:02.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:17:02.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:17:02.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215111 2026-02-17T06:17:02.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:17:02.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215111 2026-02-17T06:17:02.143 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 103079215111 2026-02-17T06:17:02.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215111' 2026-02-17T06:17:02.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:17:02.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215111 2026-02-17T06:17:02.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:17:03.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:17:03.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:17:03.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215111 -lt 103079215111 2026-02-17T06:17:03.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:17:03.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-85899345928 2026-02-17T06:17:03.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:17:03.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:17:03.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-85899345928 2026-02-17T06:17:03.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:17:03.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345928 2026-02-17T06:17:03.463 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 85899345928 2026-02-17T06:17:03.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 85899345928' 2026-02-17T06:17:03.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:17:03.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345928 -lt 85899345928 2026-02-17T06:17:03.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:706: TEST_repair_stats: ceph pg 1.0 query 2026-02-17T06:17:03.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:706: TEST_repair_stats: jq .info.stats.stat_sum 2026-02-17T06:17:03.684 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_bytes": 210, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects": 30, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_object_clones": 0, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_object_copies": 60, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_missing": 0, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_degraded": 0, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_misplaced": 0, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_unfound": 0, 2026-02-17T06:17:03.685 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_dirty": 30, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_whiteouts": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_read": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_read_kb": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_write": 30, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_write_kb": 30, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrub_errors": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_deep_scrub_errors": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_recovered": 20, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_bytes_recovered": 140, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_keys_recovered": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_omap": 0, 2026-02-17T06:17:03.686 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush_kb": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict_kb": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_promote": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush_mode_high": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush_mode_low": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict_mode_some": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict_mode_full": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_pinned": 0, 2026-02-17T06:17:03.687 INFO:tasks.workunit.client.0.trial074.stdout: "num_legacy_snapsets": 0, 2026-02-17T06:17:03.688 INFO:tasks.workunit.client.0.trial074.stdout: "num_large_omap_objects": 0, 2026-02-17T06:17:03.688 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_manifest": 0, 2026-02-17T06:17:03.688 INFO:tasks.workunit.client.0.trial074.stdout: "num_omap_bytes": 0, 2026-02-17T06:17:03.688 INFO:tasks.workunit.client.0.trial074.stdout: "num_omap_keys": 0, 2026-02-17T06:17:03.688 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_repaired": 20 2026-02-17T06:17:03.688 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:17:03.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:707: TEST_repair_stats: ceph pg 1.0 query 2026-02-17T06:17:03.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:707: TEST_repair_stats: jq .info.stats.stat_sum.num_objects_repaired 2026-02-17T06:17:03.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:707: TEST_repair_stats: COUNT=20 2026-02-17T06:17:03.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:708: TEST_repair_stats: test 20 = 20 2026-02-17T06:17:03.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:710: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T06:17:03.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:710: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 1 )' 2026-02-17T06:17:03.898 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "up_from": 20, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 85899345928, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "num_pgs": 1, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "num_osds": 1, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_osds": 1, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "kb": 104857600, 2026-02-17T06:17:03.909 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used": 28568, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_data": 328, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_omap": 4, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_meta": 28219, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "kb_avail": 104829032, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "statfs": { 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "total": 107374182400, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "available": 107344928768, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "internally_reserved": 0, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "allocated": 335872, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "data_stored": 56189, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed": 0, 2026-02-17T06:17:03.910 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_allocated": 0, 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_original": 0, 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "omap_allocated": 4393, 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "internal_metadata": 28896983 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "hb_peers": [ 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: 0 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "snap_trim_queue_len": 0, 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "num_snap_trimming": 0, 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "num_shards_repaired": 10, 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "op_queue_age_hist": { 2026-02-17T06:17:03.911 INFO:tasks.workunit.client.0.trial074.stdout: "histogram": [], 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: "upper_bound": 1 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: "perf_stat": { 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ms": 0, 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ms": 0, 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ns": 0, 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ns": 0 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout: "alerts": [] 2026-02-17T06:17:03.912 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:17:03.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:711: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T06:17:03.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:711: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 1 ).num_shards_repaired' 2026-02-17T06:17:04.047 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:17:04.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:711: TEST_repair_stats: COUNT=10 2026-02-17T06:17:04.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:712: TEST_repair_stats: expr 20 / 2 2026-02-17T06:17:04.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:712: TEST_repair_stats: test 10 = 10 2026-02-17T06:17:04.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:714: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T06:17:04.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:714: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 0 )' 2026-02-17T06:17:04.197 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:17:04.205 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:17:04.205 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "up_from": 24, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 103079215111, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "num_pgs": 1, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "num_osds": 1, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_osds": 1, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "kb": 104857600, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used": 28568, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_data": 328, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_omap": 4, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_meta": 28219, 2026-02-17T06:17:04.206 INFO:tasks.workunit.client.0.trial074.stdout: "kb_avail": 104829032, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "statfs": { 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "total": 107374182400, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "available": 107344928768, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "internally_reserved": 0, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "allocated": 335872, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "data_stored": 56189, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed": 0, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_allocated": 0, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_original": 0, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "omap_allocated": 4398, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "internal_metadata": 28896978 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:04.207 INFO:tasks.workunit.client.0.trial074.stdout: "hb_peers": [ 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: "snap_trim_queue_len": 0, 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: "num_snap_trimming": 0, 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: "num_shards_repaired": 10, 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: "op_queue_age_hist": { 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: "histogram": [], 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: "upper_bound": 1 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:04.208 INFO:tasks.workunit.client.0.trial074.stdout: "perf_stat": { 2026-02-17T06:17:04.209 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ms": 0, 2026-02-17T06:17:04.209 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ms": 0, 2026-02-17T06:17:04.209 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ns": 0, 2026-02-17T06:17:04.209 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ns": 0 2026-02-17T06:17:04.209 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:04.209 INFO:tasks.workunit.client.0.trial074.stdout: "alerts": [] 2026-02-17T06:17:04.209 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:17:04.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:715: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T06:17:04.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:715: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 0 ).num_shards_repaired' 2026-02-17T06:17:04.346 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:17:04.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:715: TEST_repair_stats: COUNT=10 2026-02-17T06:17:04.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:716: TEST_repair_stats: expr 20 / 2 2026-02-17T06:17:04.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:716: TEST_repair_stats: test 10 = 10 2026-02-17T06:17:04.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:718: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T06:17:04.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:718: TEST_repair_stats: jq .pg_map.osd_stats_sum 2026-02-17T06:17:04.497 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "up_from": 0, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "num_pgs": 2, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "num_osds": 2, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_osds": 2, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_omap_osds": 2, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "kb": 209715200, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used": 57136, 2026-02-17T06:17:04.507 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_data": 656, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_omap": 8, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_meta": 56439, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "kb_avail": 209658064, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "statfs": { 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "total": 214748364800, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "available": 214689857536, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "internally_reserved": 0, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "allocated": 671744, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "data_stored": 112378, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed": 0, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_allocated": 0, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_original": 0, 2026-02-17T06:17:04.508 INFO:tasks.workunit.client.0.trial074.stdout: "omap_allocated": 8791, 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "internal_metadata": 57793961 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "hb_peers": [], 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "snap_trim_queue_len": 0, 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "num_snap_trimming": 0, 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "num_shards_repaired": 20, 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "op_queue_age_hist": { 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "histogram": [], 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: "upper_bound": 1 2026-02-17T06:17:04.509 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: "perf_stat": { 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ms": 0, 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ms": 0, 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ns": 0, 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ns": 0 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: "alerts": [], 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout: "network_ping_times": [] 2026-02-17T06:17:04.510 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:17:04.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:719: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T06:17:04.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:719: TEST_repair_stats: jq .pg_map.osd_stats_sum.num_shards_repaired 2026-02-17T06:17:04.651 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:17:04.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:719: TEST_repair_stats: COUNT=20 2026-02-17T06:17:04.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:720: TEST_repair_stats: test 20 = 20 2026-02-17T06:17:04.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:17:04.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:17:04.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:17:04.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:17:04.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:17:04.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:17:04.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:17:04.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:17:04.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:17:04.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:17:04.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:17:04.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:17:04.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:17:04.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:17:04.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:17:04.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:17:04.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:17:04.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:17:04.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:17:04.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:17:04.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:17:04.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:17:04.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:17:04.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:17:04.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:04.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:04.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:17:04.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:17:04.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:17:04.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:17:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:17:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:17:04.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:17:04.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:17:04.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:17:04.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:17:04.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:17:04.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:17:04.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:17:04.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:17:04.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:17:04.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:17:04.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:17:04.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:17:04.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:17:04.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:17:04.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:04.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:04.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:17:04.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:17:04.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:17:04.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:17:04.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:17:04.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:04.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:04.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:17:04.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:17:04.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:17:04.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:17:04.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:17:04.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_repair_stats_ec td/osd-scrub-repair 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:724: TEST_repair_stats_ec: local dir=td/osd-scrub-repair 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:725: TEST_repair_stats_ec: local poolname=testpool 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:726: TEST_repair_stats_ec: local OSDS=3 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:727: TEST_repair_stats_ec: local OBJS=30 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:729: TEST_repair_stats_ec: local REPAIRS=26 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:730: TEST_repair_stats_ec: local allow_overwrites=false 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:733: TEST_repair_stats_ec: run_mon td/osd-scrub-repair a 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:17:04.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:17:04.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:17:04.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:17:04.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:17:04.837 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:17:04.838 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:04.838 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:04.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:04.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:17:04.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:17:04.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:17:04.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:17:04.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:17:04.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:17:04.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:17:04.868 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:17:04.869 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:17:04.869 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:17:04.870 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:17:04.870 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:04.870 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:04.870 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:17:04.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:17:04.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:17:04.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:17:04.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:17:04.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:17:04.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:17:04.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:17:04.927 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:17:04.927 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:17:04.927 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:17:04.927 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:17:04.927 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:04.928 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:04.928 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:17:04.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:17:04.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:17:04.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:734: TEST_repair_stats_ec: run_mgr td/osd-scrub-repair x 2026-02-17T06:17:04.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:17:04.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:17:04.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:17:04.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:17:04.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:17:04.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:17:05.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:17:05.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:17:05.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:17:05.076 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:17:05.077 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:05.077 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:05.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:05.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:17:05.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:17:05.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats_ec: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:17:05.097 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: expr 3 - 1 2026-02-17T06:17:05.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: seq 0 2 2026-02-17T06:17:05.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:17:05.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:05.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:17:05.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:17:05.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:17:05.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2e01c4e3-0fad-4b04-9608-577f1225444f 2026-02-17T06:17:05.107 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 2e01c4e3-0fad-4b04-9608-577f1225444f 2026-02-17T06:17:05.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2e01c4e3-0fad-4b04-9608-577f1225444f' 2026-02-17T06:17:05.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:17:05.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDhB5RpK8mhBhAA/k6+lJc5lfOjMuUrB3vCoQ== 2026-02-17T06:17:05.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDhB5RpK8mhBhAA/k6+lJc5lfOjMuUrB3vCoQ=="}' 2026-02-17T06:17:05.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2e01c4e3-0fad-4b04-9608-577f1225444f -i td/osd-scrub-repair/0/new.json 2026-02-17T06:17:05.202 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:17:05.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:17:05.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDhB5RpK8mhBhAA/k6+lJc5lfOjMuUrB3vCoQ== --osd-uuid 2e01c4e3-0fad-4b04-9608-577f1225444f 2026-02-17T06:17:05.228 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:05.224+0000 7fd3a8cfc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:05.229 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:05.224+0000 7fd3a8cfc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:05.230 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:05.224+0000 7fd3a8cfc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:05.231 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:05.224+0000 7fd3a8cfc8c0 -1 bdev(0x558a41c01c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:17:05.231 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:05.224+0000 7fd3a8cfc8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:17:07.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:17:07.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:17:07.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:17:07.869 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:17:07.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:17:08.053 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:17:08.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:17:08.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:17:08.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:17:08.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:17:08.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:17:08.069 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:08.064+0000 7f5975ba48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:08.070 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:08.064+0000 7f5975ba48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:08.071 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:08.068+0000 7f5975ba48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:17:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:17:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:17:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:17:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:17:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:17:08.201 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:17:08.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:08.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:17:08.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:09.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:09.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:09.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:17:09.356 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:17:09.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:09.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:17:09.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:09.573 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:09.568+0000 7f5975ba48c0 -1 Falling back to public interface 2026-02-17T06:17:10.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:10.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:10.512 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:17:10.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:17:10.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:10.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:17:10.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:11.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:11.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:11.663 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:17:11.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:17:11.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:11.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:17:11.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:12.560 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:12.556+0000 7f5975ba48c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:17:12.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:12.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:12.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:17:12.818 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:17:12.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:12.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:17:12.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:13.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:13.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:13.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:17:13.971 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:17:13.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:13.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:17:14.123 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1331191394,v1:127.0.0.1:6803/1331191394] [v2:127.0.0.1:6804/1331191394,v1:127.0.0.1:6805/1331191394] exists,up 2e01c4e3-0fad-4b04-9608-577f1225444f 2026-02-17T06:17:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:17:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:17:14.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:17:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:14.128 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:17:14.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:17:14.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:17:14.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:17:14.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=254e4cba-9c65-44f9-a6c6-91c09e1629a2 2026-02-17T06:17:14.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 254e4cba-9c65-44f9-a6c6-91c09e1629a2' 2026-02-17T06:17:14.131 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 254e4cba-9c65-44f9-a6c6-91c09e1629a2 2026-02-17T06:17:14.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:17:14.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDqB5RpykY8CBAAWt6HnUIBxuv2RB4EeR2wcg== 2026-02-17T06:17:14.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDqB5RpykY8CBAAWt6HnUIBxuv2RB4EeR2wcg=="}' 2026-02-17T06:17:14.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 254e4cba-9c65-44f9-a6c6-91c09e1629a2 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:17:14.298 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:17:14.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:17:14.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDqB5RpykY8CBAAWt6HnUIBxuv2RB4EeR2wcg== --osd-uuid 254e4cba-9c65-44f9-a6c6-91c09e1629a2 2026-02-17T06:17:14.333 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:14.328+0000 7efd3d8218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:14.334 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:14.328+0000 7efd3d8218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:14.336 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:14.332+0000 7efd3d8218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:14.336 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:14.332+0000 7efd3d8218c0 -1 bdev(0x555a232bfc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:17:14.336 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:14.332+0000 7efd3d8218c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:17:16.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:17:16.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:17:16.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:17:16.639 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:17:16.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:17:16.827 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:17:16.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:17:16.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:17:16.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:17:16.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:17:16.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:17:16.853 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:16.848+0000 7f2e9c51b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:16.854 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:16.848+0000 7f2e9c51b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:16.855 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:16.852+0000 7f2e9c51b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:16.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:17:16.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:17:16.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:17:16.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:17:16.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:17:16.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:16.984 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:17:16.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:17:16.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:16.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:17:17.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:18.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:18.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:18.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:17:18.135 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:17:18.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:18.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:17:18.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:18.369 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:18.364+0000 7f2e9c51b8c0 -1 Falling back to public interface 2026-02-17T06:17:19.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:19.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:19.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:17:19.290 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:17:19.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:19.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:17:19.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:20.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:20.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:20.443 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:17:20.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:17:20.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:20.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:17:20.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:21.339 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:21.336+0000 7f2e9c51b8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:17:21.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:21.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:21.597 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:17:21.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:17:21.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:21.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:17:21.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:22.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:22.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:22.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:17:22.752 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:17:22.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:22.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:17:22.902 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3244035566,v1:127.0.0.1:6811/3244035566] [v2:127.0.0.1:6812/3244035566,v1:127.0.0.1:6813/3244035566] exists,up 254e4cba-9c65-44f9-a6c6-91c09e1629a2 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 2 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:17:22.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:17:22.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:17:22.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:17:22.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:17:22.904 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:17:22.904 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:17:22.904 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:17:22.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:17:22.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:17:22.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:17:22.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=803120dc-249f-4e91-a10f-f6efbddb0450 2026-02-17T06:17:22.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 803120dc-249f-4e91-a10f-f6efbddb0450' 2026-02-17T06:17:22.907 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 803120dc-249f-4e91-a10f-f6efbddb0450 2026-02-17T06:17:22.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:17:22.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDyB5Rpa7azNhAArz2EnBld5fYeoO/3Er8KiA== 2026-02-17T06:17:22.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDyB5Rpa7azNhAArz2EnBld5fYeoO/3Er8KiA=="}' 2026-02-17T06:17:22.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 803120dc-249f-4e91-a10f-f6efbddb0450 -i td/osd-scrub-repair/2/new.json 2026-02-17T06:17:23.074 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:17:23.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T06:17:23.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDyB5Rpa7azNhAArz2EnBld5fYeoO/3Er8KiA== --osd-uuid 803120dc-249f-4e91-a10f-f6efbddb0450 2026-02-17T06:17:23.101 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:23.096+0000 7fc2d13c08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:23.103 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:23.096+0000 7fc2d13c08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:23.104 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:23.100+0000 7fc2d13c08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:23.104 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:23.100+0000 7fc2d13c08c0 -1 bdev(0x55983464dc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:17:23.104 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:23.100+0000 7fc2d13c08c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T06:17:25.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T06:17:25.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:17:25.417 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:17:25.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:17:25.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:17:25.608 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:17:25.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:17:25.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:17:25.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:17:25.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:17:25.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:17:25.632 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:25.628+0000 7f6087bfd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:25.634 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:25.628+0000 7f6087bfd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:25.635 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:25.628+0000 7f6087bfd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:17:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:17:25.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:25.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:17:25.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:26.916 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:17:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:17:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:17:27.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:27.129 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:27.124+0000 7f6087bfd8c0 -1 Falling back to public interface 2026-02-17T06:17:28.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:28.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:28.066 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:17:28.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:17:28.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:28.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:17:28.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:29.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:29.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:29.222 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:17:29.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:17:29.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:29.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:17:29.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:30.081 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:17:30.076+0000 7f6087bfd8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:17:30.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:30.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:30.374 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:17:30.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:17:30.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:30.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:17:30.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:17:31.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:17:31.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:17:31.529 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:17:31.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:17:31.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:17:31.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:17:31.675 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3785017195,v1:127.0.0.1:6819/3785017195] [v2:127.0.0.1:6820/3785017195,v1:127.0.0.1:6821/3785017195] exists,up 803120dc-249f-4e91-a10f-f6efbddb0450 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:742: TEST_repair_stats_ec: create_ec_pool testpool false k=2 m=1 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=testpool 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T06:17:31.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T06:17:31.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool testpool 1 1 erasure myprofile 2026-02-17T06:17:31.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 erasure myprofile 2026-02-17T06:17:32.176 INFO:tasks.workunit.client.0.trial074.stderr:pool 'testpool' created 2026-02-17T06:17:32.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:17:33.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T06:17:33.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T06:17:33.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:17:33.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:17:33.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:17:33.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:17:33.190 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:17:33.190 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:17:33.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:17:33.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:17:33.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:17:33.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:17:33.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:17:33.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:17:33.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:17:33.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:17:33.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:17:33.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:17:33.453 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:17:33.454 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:17:33.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:17:33.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:17:33.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:17:33.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T06:17:33.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T06:17:33.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T06:17:33.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:17:33.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:17:33.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T06:17:33.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T06:17:33.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-17T06:17:33.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:17:33.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:17:33.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574850 2026-02-17T06:17:33.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574850 2026-02-17T06:17:33.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-55834574850' 2026-02-17T06:17:33.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:17:33.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T06:17:33.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:17:33.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:17:33.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T06:17:33.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:17:33.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T06:17:33.673 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T06:17:33.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T06:17:33.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:17:33.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:17:33.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:17:34.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:17:34.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:17:34.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:17:34.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:17:35.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:17:35.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:17:36.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T06:17:36.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:17:36.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T06:17:36.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:17:36.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:17:36.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T06:17:36.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:17:36.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T06:17:36.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T06:17:36.148 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705668 2026-02-17T06:17:36.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:17:36.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T06:17:36.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:17:36.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574850 2026-02-17T06:17:36.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:17:36.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:17:36.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574850 2026-02-17T06:17:36.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:17:36.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574850 2026-02-17T06:17:36.306 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574850 2026-02-17T06:17:36.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574850' 2026-02-17T06:17:36.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:17:36.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574850 -lt 55834574850 2026-02-17T06:17:36.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:17:36.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:17:36.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:17:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:17:36.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:17:36.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:17:36.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:17:36.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:17:36.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:17:36.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:17:36.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:17:36.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:17:36.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:17:36.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:17:36.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:17:36.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:17:36.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:17:36.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:17:36.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T06:17:36.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:745: TEST_repair_stats_ec: local payload=ABCDEF 2026-02-17T06:17:36.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:746: TEST_repair_stats_ec: echo ABCDEF 2026-02-17T06:17:36.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: seq 1 30 2026-02-17T06:17:36.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:36.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj11 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj12 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj13 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj14 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj15 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj16 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj17 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj18 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj19 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj20 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj21 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj22 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj23 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj24 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj25 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj26 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj27 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj28 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj29 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T06:17:37.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj30 td/osd-scrub-repair/ORIGINAL 2026-02-17T06:17:37.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:754: TEST_repair_stats_ec: get_not_primary testpool obj1 2026-02-17T06:17:37.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T06:17:37.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T06:17:37.693 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool obj1 2026-02-17T06:17:37.693 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T06:17:37.693 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:17:37.693 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T06:17:37.694 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:17:37.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T06:17:37.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool obj1 2026-02-17T06:17:37.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T06:17:37.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:754: TEST_repair_stats_ec: local other=0 2026-02-17T06:17:37.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:755: TEST_repair_stats_ec: get_pg testpool obj1 2026-02-17T06:17:38.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T06:17:38.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T06:17:38.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T06:17:38.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:17:38.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:755: TEST_repair_stats_ec: local pgid=1.0 2026-02-17T06:17:38.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:756: TEST_repair_stats_ec: get_primary testpool obj1 2026-02-17T06:17:38.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T06:17:38.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:17:38.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T06:17:38.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:17:38.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:756: TEST_repair_stats_ec: local primary=1 2026-02-17T06:17:38.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:758: TEST_repair_stats_ec: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:17:38.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:17:38.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:17:38.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:17:38.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:17:38.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:17:39.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:17:39.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:759: TEST_repair_stats_ec: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T06:17:39.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:17:39.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:17:39.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:17:39.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:17:39.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:17:40.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:17:40.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: seq 1 26 2026-02-17T06:17:40.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:40.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 1 % 2 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 remove 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:40.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 remove 2026-02-17T06:17:41.584 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:ff7b1f36:::obj1:head# 2026-02-17T06:17:42.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:42.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 2 % 2 2026-02-17T06:17:42.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:42.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T06:17:42.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:42.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:42.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:42.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:42.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:42.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T06:17:42.785 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:104778fc:::obj2:head# 2026-02-17T06:17:43.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:43.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 3 % 2 2026-02-17T06:17:43.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj3 remove 2026-02-17T06:17:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:43.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj3 remove 2026-02-17T06:17:43.956 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:8dd16f86:::obj3:head# 2026-02-17T06:17:44.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:44.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 4 % 2 2026-02-17T06:17:44.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:44.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj4 remove 2026-02-17T06:17:44.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:44.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:44.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:44.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:44.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:44.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj4 remove 2026-02-17T06:17:45.160 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:0ee9ae15:::obj4:head# 2026-02-17T06:17:45.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:45.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 5 % 2 2026-02-17T06:17:45.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:45.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj5 remove 2026-02-17T06:17:45.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:45.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:45.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:45.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:45.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:45.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj5 remove 2026-02-17T06:17:46.332 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:c52c9666:::obj5:head# 2026-02-17T06:17:46.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:46.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 6 % 2 2026-02-17T06:17:46.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:46.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj6 remove 2026-02-17T06:17:46.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:46.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:46.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:46.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:46.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:46.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj6 remove 2026-02-17T06:17:47.544 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:233a42c1:::obj6:head# 2026-02-17T06:17:48.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:48.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 7 % 2 2026-02-17T06:17:48.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj7 remove 2026-02-17T06:17:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:48.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj7 remove 2026-02-17T06:17:48.712 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:9a09113a:::obj7:head# 2026-02-17T06:17:49.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:49.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 8 % 2 2026-02-17T06:17:49.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj8 remove 2026-02-17T06:17:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:49.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj8 remove 2026-02-17T06:17:49.877 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:a557efb1:::obj8:head# 2026-02-17T06:17:50.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:50.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 9 % 2 2026-02-17T06:17:50.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:50.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj9 remove 2026-02-17T06:17:50.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:50.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:50.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:50.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:50.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:50.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj9 remove 2026-02-17T06:17:51.048 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:fe351e27:::obj9:head# 2026-02-17T06:17:51.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:51.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 10 % 2 2026-02-17T06:17:51.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:51.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj10 remove 2026-02-17T06:17:51.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:51.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:51.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:51.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:51.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:51.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj10 remove 2026-02-17T06:17:52.220 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:d1337354:::obj10:head# 2026-02-17T06:17:52.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:52.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 11 % 2 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj11 remove 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:52.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj11 remove 2026-02-17T06:17:53.404 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:eeae2a94:::obj11:head# 2026-02-17T06:17:53.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:53.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 12 % 2 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj12 remove 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:53.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj12 remove 2026-02-17T06:17:54.576 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:af5f95cb:::obj12:head# 2026-02-17T06:17:55.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:55.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 13 % 2 2026-02-17T06:17:55.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:55.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj13 remove 2026-02-17T06:17:55.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:55.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:55.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:55.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:55.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:55.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj13 remove 2026-02-17T06:17:55.740 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:a61ad63e:::obj13:head# 2026-02-17T06:17:56.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:56.277 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 14 % 2 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj14 remove 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:56.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj14 remove 2026-02-17T06:17:56.924 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:e0a44829:::obj14:head# 2026-02-17T06:17:57.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:57.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 15 % 2 2026-02-17T06:17:57.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:57.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj15 remove 2026-02-17T06:17:57.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:57.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:57.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:57.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:57.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:57.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj15 remove 2026-02-17T06:17:58.095 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:ab946124:::obj15:head# 2026-02-17T06:17:58.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:58.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 16 % 2 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj16 remove 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:17:58.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj16 remove 2026-02-17T06:17:59.288 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:461f8b5e:::obj16:head# 2026-02-17T06:17:59.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:17:59.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 17 % 2 2026-02-17T06:17:59.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:17:59.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj17 remove 2026-02-17T06:17:59.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:17:59.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:17:59.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:17:59.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:17:59.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:17:59.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj17 remove 2026-02-17T06:18:00.456 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:24fc6e92:::obj17:head# 2026-02-17T06:18:01.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:01.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 18 % 2 2026-02-17T06:18:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:18:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj18 remove 2026-02-17T06:18:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:01.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:01.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:18:01.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:01.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:18:01.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj18 remove 2026-02-17T06:18:01.653 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:b9836a99:::obj18:head# 2026-02-17T06:18:02.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:02.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 19 % 2 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj19 remove 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:18:02.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj19 remove 2026-02-17T06:18:02.823 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:00b31b6c:::obj19:head# 2026-02-17T06:18:03.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:03.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 20 % 2 2026-02-17T06:18:03.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:18:03.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj20 remove 2026-02-17T06:18:03.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:03.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:03.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:18:03.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:03.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:18:03.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj20 remove 2026-02-17T06:18:03.998 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:24b5611b:::obj20:head# 2026-02-17T06:18:04.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:04.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 21 % 2 2026-02-17T06:18:04.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:18:04.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj21 remove 2026-02-17T06:18:04.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:04.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:04.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:18:04.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:04.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:18:04.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj21 remove 2026-02-17T06:18:05.184 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:1243fe1a:::obj21:head# 2026-02-17T06:18:05.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:05.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 22 % 2 2026-02-17T06:18:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:18:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj22 remove 2026-02-17T06:18:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:18:05.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:05.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:18:05.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj22 remove 2026-02-17T06:18:06.351 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:c0c49def:::obj22:head# 2026-02-17T06:18:06.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:06.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 23 % 2 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj23 remove 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:18:06.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj23 remove 2026-02-17T06:18:07.523 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:242c40e0:::obj23:head# 2026-02-17T06:18:08.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:08.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 24 % 2 2026-02-17T06:18:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:18:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj24 remove 2026-02-17T06:18:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:08.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:18:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:18:08.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj24 remove 2026-02-17T06:18:08.707 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:7eacede5:::obj24:head# 2026-02-17T06:18:09.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:09.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 25 % 2 2026-02-17T06:18:09.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T06:18:09.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj25 remove 2026-02-17T06:18:09.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:09.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:09.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:18:09.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:09.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:18:09.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj25 remove 2026-02-17T06:18:09.887 INFO:tasks.workunit.client.0.trial074.stdout:remove 0#1:7109478f:::obj25:head# 2026-02-17T06:18:10.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T06:18:10.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 26 % 2 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj26 remove 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:18:10.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj26 remove 2026-02-17T06:18:11.076 INFO:tasks.workunit.client.0.trial074.stdout:remove 1#1:f3c2593a:::obj26:head# 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:766: TEST_repair_stats_ec: activate_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:18:11.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:18:11.614 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:18:11.614 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:11.614 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:11.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:18:11.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:18:11.616 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:18:11.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:18:11.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:18:11.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T06:18:11.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:18:11.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:18:11.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:18:11.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:18:11.640 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:11.635+0000 7efc0dd6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:11.642 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:11.635+0000 7efc0dd6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:11.643 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:11.635+0000 7efc0dd6c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:18:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:18:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:18:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:18:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:18:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:11.770 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:18:11.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:18:11.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:11.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:18:11.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:12.905 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:12.899+0000 7efc0dd6c8c0 -1 Falling back to public interface 2026-02-17T06:18:12.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:12.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:12.922 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:18:12.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:18:12.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:12.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:18:13.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:14.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:14.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:14.076 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:18:14.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:18:14.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:14.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:18:14.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:15.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:15.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:15.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:18:15.231 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:18:15.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:15.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:18:15.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:15.929 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:15.923+0000 7efc0dd6c8c0 -1 osd.1 20 log_to_monitors true 2026-02-17T06:18:16.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:16.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:16.387 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:18:16.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:18:16.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:16.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:18:16.537 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 24 up_thru 24 down_at 21 last_clean_interval [9,20) [v2:127.0.0.1:6802/1906050666,v1:127.0.0.1:6803/1906050666] [v2:127.0.0.1:6804/1906050666,v1:127.0.0.1:6805/1906050666] exists,up 254e4cba-9c65-44f9-a6c6-91c09e1629a2 2026-02-17T06:18:16.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:18:16.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:18:16.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:18:16.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:767: TEST_repair_stats_ec: activate_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:18:16.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:18:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:18:16.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:18:16.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:18:16.545 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:18:16.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T06:18:16.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:18:16.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:18:16.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:18:16.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:18:16.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:18:16.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:16.559+0000 7fd345ea08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:16.566 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:16.559+0000 7fd345ea08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:16.567 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:16.559+0000 7fd345ea08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:16.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:16.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:17.797 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:17.791+0000 7fd345ea08c0 -1 Falling back to public interface 2026-02-17T06:18:17.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:17.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:17.851 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:18:17.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:18:17.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:17.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:17.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:19.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:19.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:19.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:18:19.001 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:18:19.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:19.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:19.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:20.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:20.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:20.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:18:20.157 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:18:20.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:20.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:20.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:20.808 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:20.803+0000 7fd345ea08c0 -1 osd.0 18 log_to_monitors true 2026-02-17T06:18:21.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:21.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:21.310 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:18:21.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:18:21.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:21.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:21.461 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 28 up_thru 0 down_at 19 last_clean_interval [5,18) [v2:127.0.0.1:6810/1916461986,v1:127.0.0.1:6811/1916461986] [v2:127.0.0.1:6812/1916461986,v1:127.0.0.1:6813/1916461986] exists,up 2e01c4e3-0fad-4b04-9608-577f1225444f 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:768: TEST_repair_stats_ec: wait_for_clean 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:18:21.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:18:21.463 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:18:21.463 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:18:21.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:18:21.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:18:21.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:18:21.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:18:21.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:18:21.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:18:21.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:18:21.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:18:21.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:18:21.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:18:21.733 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:18:21.733 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:18:21.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:18:21.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:21.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:18:21.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084290 2026-02-17T06:18:21.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084290 2026-02-17T06:18:21.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084290' 2026-02-17T06:18:21.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:21.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:18:21.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215107 2026-02-17T06:18:21.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215107 2026-02-17T06:18:21.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084290 1-103079215107' 2026-02-17T06:18:21.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:21.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:18:21.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574861 2026-02-17T06:18:21.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574861 2026-02-17T06:18:21.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084290 1-103079215107 2-55834574861' 2026-02-17T06:18:21.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:21.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-120259084290 2026-02-17T06:18:21.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:21.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:18:21.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-120259084290 2026-02-17T06:18:21.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:21.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084290 2026-02-17T06:18:21.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 120259084290' 2026-02-17T06:18:21.954 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 120259084290 2026-02-17T06:18:21.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:22.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 120259084290 2026-02-17T06:18:22.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:18:23.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:23.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 120259084290 2026-02-17T06:18:23.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:24.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:18:24.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:24.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084290 -lt 120259084290 2026-02-17T06:18:24.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:24.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-103079215107 2026-02-17T06:18:24.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:24.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:18:24.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-103079215107 2026-02-17T06:18:24.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:24.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215107 2026-02-17T06:18:24.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 103079215107' 2026-02-17T06:18:24.432 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 103079215107 2026-02-17T06:18:24.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:18:24.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215107 -lt 103079215107 2026-02-17T06:18:24.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:24.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574861 2026-02-17T06:18:24.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:24.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:18:24.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574861 2026-02-17T06:18:24.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:24.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574861 2026-02-17T06:18:24.588 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574861 2026-02-17T06:18:24.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574861' 2026-02-17T06:18:24.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:18:24.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574861 -lt 55834574861 2026-02-17T06:18:24.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:18:24.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:18:24.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:18:24.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:18:24.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:18:24.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:18:24.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:18:24.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:18:24.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:18:24.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:18:24.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:18:25.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:18:25.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:18:25.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:18:25.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:770: TEST_repair_stats_ec: repair 1.0 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:18:25.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:25.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:25.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:18:25.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:25.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T06:18:25.532 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:18:25.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:25.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:25.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:18:25.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:17:32.173384+0000 '>' 2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:25.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:18:26.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:18:26.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:18:26.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:18:26.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:18:26.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:26.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:26.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:18:26.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:17:32.173384+0000 '>' 2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:26.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:18:27.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:18:27.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:18:27.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:18:27.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:18:27.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:27.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:27.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:18:27.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:17:32.173384+0000 '>' 2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:27.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:18:28.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:18:28.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:18:28.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:18:28.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:18:28.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:28.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:28.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:18:29.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:17:32.173384+0000 '>' 2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:29.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:18:30.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:18:30.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:18:30.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:18:30.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:18:30.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:30.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:30.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:18:30.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:17:32.173384+0000 '>' 2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:30.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:18:31.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:18:31.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:18:31.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:18:31.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:18:31.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:31.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:31.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:18:31.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:18:26.021538+0000 '>' 2026-02-17T06:17:32.173384+0000 2026-02-17T06:18:31.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:18:31.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:771: TEST_repair_stats_ec: wait_for_clean 2026-02-17T06:18:31.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:18:31.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:18:31.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:18:31.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:18:31.447 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:18:31.447 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:18:31.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:18:31.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:18:31.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:18:31.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:18:31.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:18:31.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:18:31.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:18:31.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:18:31.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:18:31.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:18:31.720 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:18:31.720 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:18:31.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:18:31.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:31.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:18:31.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084293 2026-02-17T06:18:31.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084293 2026-02-17T06:18:31.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084293' 2026-02-17T06:18:31.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:31.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:18:31.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215110 2026-02-17T06:18:31.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215110 2026-02-17T06:18:31.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084293 1-103079215110' 2026-02-17T06:18:31.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:31.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:18:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574864 2026-02-17T06:18:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574864 2026-02-17T06:18:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084293 1-103079215110 2-55834574864' 2026-02-17T06:18:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:31.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-120259084293 2026-02-17T06:18:31.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:31.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:18:31.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-120259084293 2026-02-17T06:18:31.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:31.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084293 2026-02-17T06:18:31.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 120259084293' 2026-02-17T06:18:31.941 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 120259084293 2026-02-17T06:18:31.941 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:32.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084292 -lt 120259084293 2026-02-17T06:18:32.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:33.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:18:33.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:33.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084292 -lt 120259084293 2026-02-17T06:18:33.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:34.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:18:34.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:34.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084293 -lt 120259084293 2026-02-17T06:18:34.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:34.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-103079215110 2026-02-17T06:18:34.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:34.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:18:34.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-103079215110 2026-02-17T06:18:34.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:34.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215110 2026-02-17T06:18:34.419 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 103079215110 2026-02-17T06:18:34.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 103079215110' 2026-02-17T06:18:34.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:18:34.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215110 -lt 103079215110 2026-02-17T06:18:34.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:34.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574864 2026-02-17T06:18:34.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:34.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:18:34.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574864 2026-02-17T06:18:34.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:34.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574864 2026-02-17T06:18:34.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-17T06:18:34.583 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574864 2026-02-17T06:18:34.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:18:34.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-17T06:18:34.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:18:34.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:18:34.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:18:34.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:18:34.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:18:34.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:18:34.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:18:34.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:18:34.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:18:34.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:18:34.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:18:35.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:18:35.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:18:35.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:18:35.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:18:35.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:18:35.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:18:35.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:18:35.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:772: TEST_repair_stats_ec: ceph pg dump pgs 2026-02-17T06:18:35.381 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T06:18:35.382 INFO:tasks.workunit.client.0.trial074.stdout:1.0 30 0 0 0 0 210 0 0 30 0 30 active+clean 2026-02-17T06:18:26.056348+0000 18'30 30:140 [1,0,2] 1 [1,0,2] 1 18'30 2026-02-17T06:18:26.021538+0000 18'30 2026-02-17T06:18:26.021538+0000 0 1 periodic scrub scheduled @ 2026-02-18T06:18:26.021538+0000 17 0 2026-02-17T06:18:35.382 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T06:18:35.382 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T06:18:35.382 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:18:35.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:773: TEST_repair_stats_ec: flush_pg_stats 2026-02-17T06:18:35.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:18:35.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:18:35.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:18:35.560 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:18:35.560 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:18:35.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:18:35.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:35.561 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:18:35.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084294 2026-02-17T06:18:35.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084294 2026-02-17T06:18:35.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084294' 2026-02-17T06:18:35.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:35.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:18:35.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215111 2026-02-17T06:18:35.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215111 2026-02-17T06:18:35.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084294 1-103079215111' 2026-02-17T06:18:35.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:35.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:18:35.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574865 2026-02-17T06:18:35.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574865 2026-02-17T06:18:35.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-120259084294 1-103079215111 2-55834574865' 2026-02-17T06:18:35.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:35.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-120259084294 2026-02-17T06:18:35.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:35.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:18:35.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-120259084294 2026-02-17T06:18:35.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:35.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084294 2026-02-17T06:18:35.790 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 120259084294 2026-02-17T06:18:35.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 120259084294' 2026-02-17T06:18:35.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:35.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084293 -lt 120259084294 2026-02-17T06:18:35.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:36.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:18:36.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:37.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084293 -lt 120259084294 2026-02-17T06:18:37.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:38.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:18:38.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:38.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084295 -lt 120259084294 2026-02-17T06:18:38.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:38.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-103079215111 2026-02-17T06:18:38.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:38.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:18:38.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-103079215111 2026-02-17T06:18:38.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:38.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215111 2026-02-17T06:18:38.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 103079215111' 2026-02-17T06:18:38.260 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 103079215111 2026-02-17T06:18:38.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:18:38.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215112 -lt 103079215111 2026-02-17T06:18:38.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:38.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574865 2026-02-17T06:18:38.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:38.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:18:38.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574865 2026-02-17T06:18:38.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:38.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574865 2026-02-17T06:18:38.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574865' 2026-02-17T06:18:38.419 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574865 2026-02-17T06:18:38.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:18:38.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574866 -lt 55834574865 2026-02-17T06:18:38.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:776: TEST_repair_stats_ec: ceph pg 1.0 query 2026-02-17T06:18:38.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:776: TEST_repair_stats_ec: jq .info.stats.stat_sum 2026-02-17T06:18:38.654 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:38.654 INFO:tasks.workunit.client.0.trial074.stdout: "num_bytes": 210, 2026-02-17T06:18:38.654 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects": 30, 2026-02-17T06:18:38.654 INFO:tasks.workunit.client.0.trial074.stdout: "num_object_clones": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_object_copies": 90, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_missing": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_degraded": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_misplaced": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_unfound": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_dirty": 30, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_whiteouts": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_read": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_read_kb": 0, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_write": 30, 2026-02-17T06:18:38.655 INFO:tasks.workunit.client.0.trial074.stdout: "num_write_kb": 30, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrub_errors": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_deep_scrub_errors": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_recovered": 26, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_bytes_recovered": 182, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_keys_recovered": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_omap": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush_kb": 0, 2026-02-17T06:18:38.656 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict_kb": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_promote": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush_mode_high": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_flush_mode_low": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict_mode_some": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_evict_mode_full": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_pinned": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_legacy_snapsets": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_large_omap_objects": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_manifest": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_omap_bytes": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_omap_keys": 0, 2026-02-17T06:18:38.657 INFO:tasks.workunit.client.0.trial074.stdout: "num_objects_repaired": 26 2026-02-17T06:18:38.658 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:38.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats_ec: ceph pg 1.0 query 2026-02-17T06:18:38.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats_ec: jq .info.stats.stat_sum.num_objects_repaired 2026-02-17T06:18:38.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats_ec: COUNT=26 2026-02-17T06:18:38.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:778: TEST_repair_stats_ec: test 26 = 26 2026-02-17T06:18:38.727 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: expr 3 - 1 2026-02-17T06:18:38.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: seq 0 2 2026-02-17T06:18:38.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:18:38.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:781: TEST_repair_stats_ec: '[' 0 = 0 -o 0 = 1 ']' 2026-02-17T06:18:38.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: expr 26 / 2 2026-02-17T06:18:38.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: repair=13 2026-02-17T06:18:38.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:38.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 0 )' 2026-02-17T06:18:38.868 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "up_from": 28, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 120259084295, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "num_pgs": 1, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "num_osds": 1, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_osds": 1, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "kb": 104857600, 2026-02-17T06:18:38.877 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used": 28484, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_data": 368, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_omap": 4, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_meta": 28091, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "kb_avail": 104829116, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "statfs": { 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "total": 107374182400, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "available": 107345014784, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "internally_reserved": 0, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "allocated": 376832, 2026-02-17T06:18:38.878 INFO:tasks.workunit.client.0.trial074.stdout: "data_stored": 206358, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed": 0, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_allocated": 0, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_original": 0, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "omap_allocated": 4399, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "internal_metadata": 28765905 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "hb_peers": [ 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: 1, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: 2 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "snap_trim_queue_len": 0, 2026-02-17T06:18:38.879 INFO:tasks.workunit.client.0.trial074.stdout: "num_snap_trimming": 0, 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "num_shards_repaired": 13, 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "op_queue_age_hist": { 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "histogram": [], 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "upper_bound": 1 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "perf_stat": { 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ms": 0, 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ms": 0, 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ns": 0, 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ns": 0 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout: "alerts": [] 2026-02-17T06:18:38.880 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:38.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:38.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 0 ).num_shards_repaired' 2026-02-17T06:18:39.020 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:39.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: COUNT=13 2026-02-17T06:18:39.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:789: TEST_repair_stats_ec: test 13 = 13 2026-02-17T06:18:39.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:18:39.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:781: TEST_repair_stats_ec: '[' 1 = 0 -o 1 = 1 ']' 2026-02-17T06:18:39.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: expr 26 / 2 2026-02-17T06:18:39.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: repair=13 2026-02-17T06:18:39.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:39.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 1 )' 2026-02-17T06:18:39.170 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "up_from": 24, 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 103079215112, 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "num_pgs": 1, 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "num_osds": 1, 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_osds": 1, 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T06:18:39.178 INFO:tasks.workunit.client.0.trial074.stdout: "kb": 104857600, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used": 28480, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_data": 368, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_omap": 4, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_meta": 28091, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "kb_avail": 104829120, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "statfs": { 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "total": 107374182400, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "available": 107345018880, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "internally_reserved": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "allocated": 376832, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "data_stored": 206358, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_allocated": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_original": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "omap_allocated": 4398, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "internal_metadata": 28765906 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "hb_peers": [ 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: 2 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "snap_trim_queue_len": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "num_snap_trimming": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "num_shards_repaired": 13, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "op_queue_age_hist": { 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "histogram": [], 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "upper_bound": 1 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "perf_stat": { 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ms": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ms": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ns": 0, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ns": 0 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout: "alerts": [] 2026-02-17T06:18:39.179 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:39.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:39.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 1 ).num_shards_repaired' 2026-02-17T06:18:39.321 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: COUNT=13 2026-02-17T06:18:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:789: TEST_repair_stats_ec: test 13 = 13 2026-02-17T06:18:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:18:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:781: TEST_repair_stats_ec: '[' 2 = 0 -o 2 = 1 ']' 2026-02-17T06:18:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:784: TEST_repair_stats_ec: repair=0 2026-02-17T06:18:39.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:39.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 2 )' 2026-02-17T06:18:39.471 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:39.479 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 2, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "up_from": 13, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 55834574866, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "num_pgs": 1, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "num_osds": 1, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_osds": 1, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "kb": 104857600, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used": 27328, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_data": 368, 2026-02-17T06:18:39.480 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_omap": 1, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_meta": 26942, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "kb_avail": 104830272, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "statfs": { 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "total": 107374182400, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "available": 107346198528, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "internally_reserved": 0, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "allocated": 376832, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "data_stored": 206358, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed": 0, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_allocated": 0, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_original": 0, 2026-02-17T06:18:39.481 INFO:tasks.workunit.client.0.trial074.stdout: "omap_allocated": 1594, 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "internal_metadata": 27589062 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "hb_peers": [ 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: 0, 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "snap_trim_queue_len": 0, 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "num_snap_trimming": 0, 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "num_shards_repaired": 0, 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "op_queue_age_hist": { 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "histogram": [], 2026-02-17T06:18:39.482 INFO:tasks.workunit.client.0.trial074.stdout: "upper_bound": 1 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: "perf_stat": { 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ms": 0, 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ms": 0, 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ns": 0, 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ns": 0 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout: "alerts": [] 2026-02-17T06:18:39.483 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:39.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:39.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 2 ).num_shards_repaired' 2026-02-17T06:18:39.621 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:39.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: COUNT=0 2026-02-17T06:18:39.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:789: TEST_repair_stats_ec: test 0 = 0 2026-02-17T06:18:39.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:792: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:39.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:792: TEST_repair_stats_ec: jq .pg_map.osd_stats_sum 2026-02-17T06:18:39.771 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:39.780 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:39.780 INFO:tasks.workunit.client.0.trial074.stdout: "up_from": 0, 2026-02-17T06:18:39.780 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "num_pgs": 3, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "num_osds": 3, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_osds": 3, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "kb": 314572800, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used": 84292, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_data": 1104, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_omap": 10, 2026-02-17T06:18:39.781 INFO:tasks.workunit.client.0.trial074.stdout: "kb_used_meta": 83125, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "kb_avail": 314488508, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "statfs": { 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "total": 322122547200, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "available": 322036232192, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "internally_reserved": 0, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "allocated": 1130496, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "data_stored": 619074, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed": 0, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_allocated": 0, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "data_compressed_original": 0, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "omap_allocated": 10391, 2026-02-17T06:18:39.782 INFO:tasks.workunit.client.0.trial074.stdout: "internal_metadata": 85120873 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "hb_peers": [], 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "snap_trim_queue_len": 0, 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "num_snap_trimming": 0, 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "num_shards_repaired": 26, 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "op_queue_age_hist": { 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "histogram": [], 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "upper_bound": 1 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "perf_stat": { 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ms": 0, 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ms": 0, 2026-02-17T06:18:39.783 INFO:tasks.workunit.client.0.trial074.stdout: "commit_latency_ns": 0, 2026-02-17T06:18:39.784 INFO:tasks.workunit.client.0.trial074.stdout: "apply_latency_ns": 0 2026-02-17T06:18:39.784 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:18:39.784 INFO:tasks.workunit.client.0.trial074.stdout: "alerts": [], 2026-02-17T06:18:39.784 INFO:tasks.workunit.client.0.trial074.stdout: "network_ping_times": [] 2026-02-17T06:18:39.784 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:39.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:793: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T06:18:39.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:793: TEST_repair_stats_ec: jq .pg_map.osd_stats_sum.num_shards_repaired 2026-02-17T06:18:39.922 INFO:tasks.workunit.client.0.trial074.stderr:dumped all 2026-02-17T06:18:39.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:793: TEST_repair_stats_ec: COUNT=26 2026-02-17T06:18:39.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:794: TEST_repair_stats_ec: test 26 = 26 2026-02-17T06:18:39.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:18:39.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:18:39.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:18:39.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:18:39.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:18:39.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:18:39.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:18:39.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:18:39.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:18:40.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:18:40.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:18:40.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:18:40.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:18:40.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:18:40.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:18:40.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:18:40.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:18:40.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:18:40.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:18:40.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:18:40.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:18:40.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:18:40.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:18:40.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:18:40.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:18:40.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:18:40.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:18:40.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:18:40.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:18:40.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:18:40.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:18:40.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:18:40.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:18:40.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:18:40.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:18:40.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:18:40.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:18:40.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:18:40.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:18:40.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:18:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:18:40.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:18:40.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:18:40.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:18:40.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:18:40.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:18:40.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:18:40.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:18:40.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:18:40.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:18:40.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:18:40.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:18:40.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:18:40.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_request_scrub_priority td/osd-scrub-repair 2026-02-17T06:18:40.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6197: TEST_request_scrub_priority: local dir=td/osd-scrub-repair 2026-02-17T06:18:40.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6198: TEST_request_scrub_priority: local poolname=psr_pool 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6199: TEST_request_scrub_priority: local objname=POBJ 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6200: TEST_request_scrub_priority: local OBJECTS=64 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6201: TEST_request_scrub_priority: local PGS=8 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6203: TEST_request_scrub_priority: run_mon td/osd-scrub-repair a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:18:40.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T06:18:40.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:18:40.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:18:40.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:18:40.114 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:18:40.114 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.114 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:40.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T06:18:40.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:18:40.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:18:40.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:18:40.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:18:40.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:18:40.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:18:40.146 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:18:40.146 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:18:40.146 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:18:40.146 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:18:40.146 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.146 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.146 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:18:40.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:18:40.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:18:40.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:18:40.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:18:40.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:18:40.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:18:40.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:18:40.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:18:40.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:18:40.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:18:40.204 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:18:40.204 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.204 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.205 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:18:40.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:18:40.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:18:40.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6204: TEST_request_scrub_priority: run_mgr td/osd-scrub-repair x 2026-02-17T06:18:40.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:18:40.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:18:40.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:18:40.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:18:40.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:18:40.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:18:40.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:18:40.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:18:40.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:18:40.353 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:18:40.353 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.353 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:40.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:18:40.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6205: TEST_request_scrub_priority: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6206: TEST_request_scrub_priority: ceph_osd_args+=--osd_scrub_backoff_ratio=0 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6207: TEST_request_scrub_priority: run_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:18:40.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:18:40.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T06:18:40.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:18:40.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:18:40.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=062f8e5a-012f-407a-8e2e-558ce6d7ddf1 2026-02-17T06:18:40.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 062f8e5a-012f-407a-8e2e-558ce6d7ddf1' 2026-02-17T06:18:40.377 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 062f8e5a-012f-407a-8e2e-558ce6d7ddf1 2026-02-17T06:18:40.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:18:40.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBACJRpVe31FhAALpGQJLFiSQLuZGD+Td/mPQ== 2026-02-17T06:18:40.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBACJRpVe31FhAALpGQJLFiSQLuZGD+Td/mPQ=="}' 2026-02-17T06:18:40.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 062f8e5a-012f-407a-8e2e-558ce6d7ddf1 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:18:40.478 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:18:40.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:18:40.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --mkfs --key AQBACJRpVe31FhAALpGQJLFiSQLuZGD+Td/mPQ== --osd-uuid 062f8e5a-012f-407a-8e2e-558ce6d7ddf1 2026-02-17T06:18:40.503 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:40.495+0000 7f63af3a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:40.504 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:40.499+0000 7f63af3a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:40.505 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:40.499+0000 7f63af3a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:40.505 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:40.499+0000 7f63af3a58c0 -1 bdev(0x555f40589c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:18:40.506 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:40.499+0000 7f63af3a58c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:18:43.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:18:43.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:18:43.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:18:43.110 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:18:43.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:18:43.305 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:18:43.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:18:43.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T06:18:43.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:18:43.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:18:43.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:18:43.330 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:43.323+0000 7f64c665a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:43.332 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:43.327+0000 7f64c665a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:43.333 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:43.327+0000 7f64c665a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:18:43.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:18:43.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:18:43.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:18:43.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:18:43.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:18:43.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:43.456 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:18:43.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:18:43.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:43.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:43.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:44.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:44.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:44.612 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:18:44.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:18:44.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:44.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:44.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:44.837 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:44.831+0000 7f64c665a8c0 -1 Falling back to public interface 2026-02-17T06:18:45.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:45.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:45.770 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:18:45.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:18:45.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:45.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:45.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:46.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:46.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:46.924 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:18:46.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:18:46.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:46.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:47.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:47.844 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:18:47.839+0000 7f64c665a8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:18:48.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:48.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:48.078 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:18:48.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:18:48.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:48.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:48.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:18:49.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:18:49.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:18:49.228 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:18:49.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:18:49.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:18:49.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:18:49.381 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3656390599,v1:127.0.0.1:6803/3656390599] [v2:127.0.0.1:6804/3656390599,v1:127.0.0.1:6805/3656390599] exists,up 062f8e5a-012f-407a-8e2e-558ce6d7ddf1 2026-02-17T06:18:49.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:18:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:18:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:18:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6209: TEST_request_scrub_priority: create_pool psr_pool 8 8 2026-02-17T06:18:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool 8 8 2026-02-17T06:18:49.579 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool' created 2026-02-17T06:18:49.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:18:50.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6210: TEST_request_scrub_priority: wait_for_clean 2026-02-17T06:18:50.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:18:50.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:18:50.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:18:50.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:18:50.593 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:18:50.593 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:18:50.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:18:50.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:18:50.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:18:50.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:18:50.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:18:50.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:18:50.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:18:50.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:18:50.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:18:50.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:18:50.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:18:50.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:18:50.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:18:50.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836482 2026-02-17T06:18:50.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836482 2026-02-17T06:18:50.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836482' 2026-02-17T06:18:50.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:18:50.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836482 2026-02-17T06:18:50.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:18:50.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:18:50.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836482 2026-02-17T06:18:50.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:18:50.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836482 2026-02-17T06:18:50.948 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836482 2026-02-17T06:18:50.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-17T06:18:50.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T06:18:51.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:52.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:18:52.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:52.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T06:18:52.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:18:53.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:18:53.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:18:53.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-17T06:18:53.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:18:53.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:18:53.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:18:53.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 == 0 2026-02-17T06:18:53.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:18:53.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:18:53.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:18:53.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:18:53.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:18:53.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:18:53.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:18:53.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=8 2026-02-17T06:18:53.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:18:53.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:18:53.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:18:53.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 8 = 8 2026-02-17T06:18:53.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6212: TEST_request_scrub_priority: local osd=0 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6213: TEST_request_scrub_priority: add_something td/osd-scrub-repair psr_pool POBJ noscrub 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=psr_pool 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=POBJ 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:18:53.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:18:54.126 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:18:54.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:18:54.329 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:18:54.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:18:54.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:18:54.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool psr_pool put POBJ td/osd-scrub-repair/ORIGINAL 2026-02-17T06:18:54.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6214: TEST_request_scrub_priority: get_primary psr_pool POBJ 2026-02-17T06:18:54.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=psr_pool 2026-02-17T06:18:54.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=POBJ 2026-02-17T06:18:54.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map psr_pool POBJ 2026-02-17T06:18:54.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:18:54.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6214: TEST_request_scrub_priority: local primary=0 2026-02-17T06:18:54.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6215: TEST_request_scrub_priority: get_pg psr_pool POBJ 2026-02-17T06:18:54.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=psr_pool 2026-02-17T06:18:54.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=POBJ 2026-02-17T06:18:54.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map psr_pool POBJ 2026-02-17T06:18:54.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:18:54.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6215: TEST_request_scrub_priority: local pg=1.1 2026-02-17T06:18:54.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: ceph osd dump 2026-02-17T06:18:54.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: grep '^pool.*['\'']psr_pool['\'']' 2026-02-17T06:18:54.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: awk '{ print $2 }' 2026-02-17T06:18:54.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: poolid=1 2026-02-17T06:18:54.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6218: TEST_request_scrub_priority: local otherpgs 2026-02-17T06:18:54.835 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: expr 8 - 1 2026-02-17T06:18:54.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: seq 0 7 2026-02-17T06:18:54.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:54.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.0 2026-02-17T06:18:54.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.0 = 1.1 ']' 2026-02-17T06:18:54.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 ' 2026-02-17T06:18:54.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:18:54.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:18:54.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:54.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:54.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:18:54.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:18:54.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.0 schedule-scrub 1.0 2026-02-17T06:18:55.052 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:55.052 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:18:55.053 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:18:55.053 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:17:15.049302+0000" 2026-02-17T06:18:55.053 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:55.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:55.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.1 2026-02-17T06:18:55.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.1 = 1.1 ']' 2026-02-17T06:18:55.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6223: TEST_request_scrub_priority: continue 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.2 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.2 = 1.1 ']' 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 ' 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:55.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:18:55.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:18:55.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.2 schedule-scrub 1.2 2026-02-17T06:18:55.280 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:55.280 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:18:55.280 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:18:55.280 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:17:15.277081+0000" 2026-02-17T06:18:55.280 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:55.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:55.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.3 2026-02-17T06:18:55.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.3 = 1.1 ']' 2026-02-17T06:18:55.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 ' 2026-02-17T06:18:55.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:18:55.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:18:55.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:55.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:55.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:18:55.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:18:55.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.3 schedule-scrub 1.3 2026-02-17T06:18:55.506 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:55.507 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:18:55.507 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:18:55.507 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:17:15.504081+0000" 2026-02-17T06:18:55.507 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.4 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.4 = 1.1 ']' 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 ' 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:55.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:55.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:18:55.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:18:55.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.4 schedule-scrub 1.4 2026-02-17T06:18:55.733 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:55.733 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:18:55.733 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:18:55.733 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:17:15.730813+0000" 2026-02-17T06:18:55.733 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.5 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.5 = 1.1 ']' 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 ' 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:55.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:55.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:18:55.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:18:55.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.5 schedule-scrub 1.5 2026-02-17T06:18:55.959 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:55.959 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:18:55.959 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:18:55.959 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:17:15.956255+0000" 2026-02-17T06:18:55.959 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:55.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:55.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.6 2026-02-17T06:18:55.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.6 = 1.1 ']' 2026-02-17T06:18:55.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 1.6 ' 2026-02-17T06:18:55.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:18:55.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:18:55.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:55.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:55.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:18:56.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:18:56.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.6 schedule-scrub 1.6 2026-02-17T06:18:56.187 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:56.187 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:18:56.187 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:18:56.187 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:17:16.183866+0000" 2026-02-17T06:18:56.187 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:56.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T06:18:56.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.7 2026-02-17T06:18:56.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.7 = 1.1 ']' 2026-02-17T06:18:56.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 1.6 1.7 ' 2026-02-17T06:18:56.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:18:56.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:18:56.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:18:56.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:18:56.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:18:56.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:18:56.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.7 schedule-scrub 1.7 2026-02-17T06:18:56.411 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:18:56.411 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:18:56.411 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:18:56.411 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:17:16.408448+0000" 2026-02-17T06:18:56.411 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:18:56.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6231: TEST_request_scrub_priority: sleep 15 2026-02-17T06:19:11.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6232: TEST_request_scrub_priority: flush_pg_stats 2026-02-17T06:19:11.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:19:11.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:19:11.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:19:11.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:19:11.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:19:11.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:19:11.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T06:19:11.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T06:19:11.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T06:19:11.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:19:11.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T06:19:11.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:19:11.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:19:11.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T06:19:11.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:19:11.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T06:19:11.654 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T06:19:11.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T06:19:11.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:11.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T06:19:11.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:12.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:19:12.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:12.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T06:19:12.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6235: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T06:19:12.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:19:12.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:12.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:12.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:19:13.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6235: TEST_request_scrub_priority: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:13.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6236: TEST_request_scrub_priority: ceph tell 1.1 scrub 2026-02-17T06:19:13.186 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:19:13.187 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:19:13.187 INFO:tasks.workunit.client.0.trial074.stdout: "must": true, 2026-02-17T06:19:13.187 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "0.000000" 2026-02-17T06:19:13.187 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:19:13.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6238: TEST_request_scrub_priority: ceph osd unset noscrub 2026-02-17T06:19:13.400 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is unset 2026-02-17T06:19:13.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6239: TEST_request_scrub_priority: ceph osd unset nodeep-scrub 2026-02-17T06:19:13.605 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is unset 2026-02-17T06:19:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6241: TEST_request_scrub_priority: wait_for_scrub 1.1 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.1 2026-02-17T06:19:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:13.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:13.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-17T06:19:13.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:19:13.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:13.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:13.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:19:13.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:18:49.575853+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:13.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:14.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:14.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:14.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-17T06:19:14.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:19:14.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:14.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:14.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:19:14.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:13.656880+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:14.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:14.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:14.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.0 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:14.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:19:14.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:14.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:19:15.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:15.049302+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:15.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:16.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:16.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:16.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:19:16.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:19:16.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:16.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:16.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:19:16.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:15.049302+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:16.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:17.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:17.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:17.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:19:17.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:19:17.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:17.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:19:17.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:17.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:15.049302+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:17.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:18.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:18.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:18.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:19:18.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:19:18.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:18.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:18.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:19:18.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:15.049302+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:18.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:19.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:19.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:19.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:19:19.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:19:19.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:19.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:19:19.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:19.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:15.049302+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:19.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:20.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:20.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:20.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:19:20.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:19:20.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:20.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:19:20.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:14.652601+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.2 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.2 2026-02-17T06:19:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.2 last_scrub_stamp 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.2 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:20.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:15.659371+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.3 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:21.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T06:19:21.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T06:19:21.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:21.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:21.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:16.656968+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.4 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.4 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.4 last_scrub_stamp 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.4 2026-02-17T06:19:21.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:21.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:21.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.4") | .last_scrub_stamp' 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:17.677448+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.5 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.5 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.5 last_scrub_stamp 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.5 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:21.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2026-02-17T06:19:21.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:18.707985+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:21.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:21.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.6 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.6 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.6 last_scrub_stamp 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.6 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:21.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2026-02-17T06:19:21.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:16.183866+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:21.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:22.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:22.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:22.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.6 last_scrub_stamp 2026-02-17T06:19:22.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.6 2026-02-17T06:19:22.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:22.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:22.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2026-02-17T06:19:22.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:16.183866+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:22.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:23.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:23.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:23.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.6 last_scrub_stamp 2026-02-17T06:19:23.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.6 2026-02-17T06:19:23.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:23.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2026-02-17T06:19:23.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:23.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:17:16.183866+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:23.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:19:24.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:19:24.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:24.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.6 last_scrub_stamp 2026-02-17T06:19:24.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.6 2026-02-17T06:19:24.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:24.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:24.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:19.684966+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.7 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.7 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:25.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-17T06:19:25.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2026-02-17T06:19:25.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:25.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:25.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:20.708530+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.1 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.1 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:19:25.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:19:25.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:19:25.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-17T06:19:25.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T06:19:25.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:19:25.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:19:25.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T06:19:25.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:19:13.656880+0000 '>' 2026-02-17T06:18:49.575853+0000 2026-02-17T06:19:25.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:19:25.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: grep 'log_channel.*scrub ok' td/osd-scrub-repair/osd.0.log 2026-02-17T06:19:25.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: grep -v purged_snaps 2026-02-17T06:19:25.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: head -1 2026-02-17T06:19:25.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: sed 's/.*[[]DBG[]]//' 2026-02-17T06:19:25.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: grep -q 1.1 2026-02-17T06:19:25.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:19:25.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:19:25.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:19:25.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:19:25.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:19:25.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:19:25.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:19:25.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:19:25.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:19:25.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:19:25.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:19:25.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:19:25.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:19:25.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:19:25.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:19:25.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:19:25.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:19:25.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:19:25.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:19:25.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:19:25.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:19:25.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:19:25.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:19:25.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:19:25.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:19:25.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:19:25.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:19:25.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:19:25.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:19:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:19:25.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:19:25.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:19:25.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:19:25.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:19:25.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:19:25.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:19:25.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:19:25.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:19:25.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:19:25.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:19:25.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:19:25.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:19:25.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:19:25.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:19:25.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:19:25.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:19:25.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:19:25.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:19:25.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:19:25.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:19:25.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:19:25.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:19:25.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:19:25.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_scrub_warning td/osd-scrub-repair 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5842: TEST_scrub_warning: local dir=td/osd-scrub-repair 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5843: TEST_scrub_warning: local poolname=psr_pool 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5844: TEST_scrub_warning: local objname=POBJ 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5845: TEST_scrub_warning: local scrubs=5 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5846: TEST_scrub_warning: local deep_scrubs=5 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5847: TEST_scrub_warning: local i1_day=86400 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5848: TEST_scrub_warning: calc 86400 '*' 7 2026-02-17T06:19:25.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 86400 * 7}' 2026-02-17T06:19:25.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5848: TEST_scrub_warning: local i7_days=604800 2026-02-17T06:19:25.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5849: TEST_scrub_warning: calc 86400 '*' 14 2026-02-17T06:19:25.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 86400 * 14}' 2026-02-17T06:19:25.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5849: TEST_scrub_warning: local i14_days=1209600 2026-02-17T06:19:25.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5850: TEST_scrub_warning: local overdue=0.5 2026-02-17T06:19:25.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5851: TEST_scrub_warning: calc 604800 + 86400 + '(' 604800 '*' 0.5 ')' 2026-02-17T06:19:25.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 604800 + 86400 + ( 604800 * 0.5 )}' 2026-02-17T06:19:25.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5851: TEST_scrub_warning: local conf_overdue_seconds=993600 2026-02-17T06:19:25.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5852: TEST_scrub_warning: calc 1209600 + 86400 + '(' 1209600 '*' 0.5 ')' 2026-02-17T06:19:25.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 1209600 + 86400 + ( 1209600 * 0.5 )}' 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5852: TEST_scrub_warning: local pool_overdue_seconds=1900800 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5854: TEST_scrub_warning: run_mon td/osd-scrub-repair a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:19:25.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T06:19:25.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:19:25.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:19:25.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:19:25.548 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:19:25.548 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.548 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:19:25.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T06:19:25.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:19:25.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:19:25.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:19:25.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:19:25.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:19:25.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:19:25.577 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:19:25.578 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:19:25.578 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:19:25.578 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:19:25.578 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.578 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.578 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:19:25.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:19:25.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:19:25.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:19:25.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:19:25.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:19:25.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:19:25.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:19:25.637 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:19:25.637 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:19:25.637 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:19:25.637 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:19:25.637 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.638 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.638 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:19:25.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:19:25.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:19:25.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5855: TEST_scrub_warning: run_mgr td/osd-scrub-repair x --mon_warn_pg_not_scrubbed_ratio=0.5 --mon_warn_pg_not_deep_scrubbed_ratio=0.5 2026-02-17T06:19:25.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:19:25.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:19:25.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:19:25.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:19:25.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:19:25.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:19:25.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:19:25.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:19:25.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:19:25.781 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:19:25.781 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.781 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:19:25.783 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:19:25.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mon_warn_pg_not_scrubbed_ratio=0.5 --mon_warn_pg_not_deep_scrubbed_ratio=0.5 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5856: TEST_scrub_warning: run_osd td/osd-scrub-repair 0 --osd_scrub_backoff_ratio=0 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:19:25.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:19:25.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_backoff_ratio=0 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:19:25.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:19:25.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2896ebb6-cf55-4fb2-9d18-7e890b4a5699 2026-02-17T06:19:25.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2896ebb6-cf55-4fb2-9d18-7e890b4a5699' 2026-02-17T06:19:25.808 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 2896ebb6-cf55-4fb2-9d18-7e890b4a5699 2026-02-17T06:19:25.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:19:25.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBtCJRpXaybMBAAnwoRObz/uXHvTjS2bG7uLQ== 2026-02-17T06:19:25.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBtCJRpXaybMBAAnwoRObz/uXHvTjS2bG7uLQ=="}' 2026-02-17T06:19:25.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2896ebb6-cf55-4fb2-9d18-7e890b4a5699 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:19:25.909 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:19:25.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:19:25.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_backoff_ratio=0 --mkfs --key AQBtCJRpXaybMBAAnwoRObz/uXHvTjS2bG7uLQ== --osd-uuid 2896ebb6-cf55-4fb2-9d18-7e890b4a5699 2026-02-17T06:19:25.934 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:25.926+0000 7fadec3c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:19:25.935 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:25.926+0000 7fadec3c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:19:25.936 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:25.930+0000 7fadec3c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:19:25.936 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:25.930+0000 7fadec3c78c0 -1 bdev(0x5630a1993c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:19:25.936 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:25.930+0000 7fadec3c78c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:19:28.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:19:28.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:19:28.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:19:28.778 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:19:28.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:19:28.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:19:28.968 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:19:28.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_backoff_ratio=0 2026-02-17T06:19:28.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:19:28.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:19:28.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:19:28.992 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:28.986+0000 7fe56517f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:19:28.994 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:28.986+0000 7fe56517f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:19:28.995 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:28.990+0000 7fe56517f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:19:29.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:19:29.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:19:29.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:19:29.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:19:29.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:19:29.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:19:29.124 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:19:29.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:19:29.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:19:29.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:19:29.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:19:30.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:19:30.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:19:30.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:19:30.275 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:19:30.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:19:30.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:19:30.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:19:30.517 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:30.510+0000 7fe56517f8c0 -1 Falling back to public interface 2026-02-17T06:19:31.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:19:31.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:19:31.427 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:19:31.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:19:31.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:19:31.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:19:31.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:19:32.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:19:32.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:19:32.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:19:32.579 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:19:32.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:19:32.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:19:32.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:19:33.504 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:19:33.494+0000 7fe56517f8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:19:33.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:19:33.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:19:33.732 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:19:33.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:19:33.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:19:33.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:19:33.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:19:34.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:19:34.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:19:34.886 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:19:34.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:19:34.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:19:34.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:19:35.036 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3696456065,v1:127.0.0.1:6803/3696456065] [v2:127.0.0.1:6804/3696456065,v1:127.0.0.1:6805/3696456065] exists,up 2896ebb6-cf55-4fb2-9d18-7e890b4a5699 2026-02-17T06:19:35.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:19:35.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:19:35.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:19:35.038 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: expr 5 + 5 2026-02-17T06:19:35.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: seq 1 10 2026-02-17T06:19:35.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:19:35.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-1 1 1 2026-02-17T06:19:35.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-1 1 1 2026-02-17T06:19:35.247 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-1' created 2026-02-17T06:19:35.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:19:36.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:19:36.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:19:36.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:19:36.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:19:36.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:19:36.261 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:19:36.261 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:19:36.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:19:36.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:19:36.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:19:36.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:19:36.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:19:36.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:19:36.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:19:36.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:19:36.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:19:36.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:19:36.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:19:36.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:19:36.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:19:36.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836482 2026-02-17T06:19:36.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836482 2026-02-17T06:19:36.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836482' 2026-02-17T06:19:36.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:19:36.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836482 2026-02-17T06:19:36.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:19:36.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:19:36.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836482 2026-02-17T06:19:36.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:19:36.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836482 2026-02-17T06:19:36.607 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836482 2026-02-17T06:19:36.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-17T06:19:36.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:36.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T06:19:36.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:37.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:19:37.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:37.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T06:19:37.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:38.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:19:38.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:39.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-17T06:19:39.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:19:39.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:39.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:39.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:19:39.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:19:39.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:19:39.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:19:39.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:19:39.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:19:39.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:19:39.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:19:39.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:19:39.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:19:39.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:39.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:39.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:19:39.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:19:39.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:19:39.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 1 = 1 ']' 2026-02-17T06:19:39.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5864: TEST_scrub_warning: ceph osd pool set psr_pool-1 scrub_max_interval 1209600 2026-02-17T06:19:39.793 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 scrub_max_interval to 1209600 2026-02-17T06:19:39.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:19:39.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 1 = 6 ']' 2026-02-17T06:19:39.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:19:39.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-2 1 1 2026-02-17T06:19:39.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-2 1 1 2026-02-17T06:19:39.997 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-2' created 2026-02-17T06:19:40.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:19:41.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:19:41.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:19:41.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:19:41.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:19:41.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:19:41.011 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:19:41.011 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:19:41.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:19:41.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:19:41.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:19:41.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:19:41.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:19:41.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:19:41.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:19:41.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:19:41.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:19:41.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:19:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:19:41.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:19:41.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:19:41.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:19:41.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:19:41.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:19:41.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:19:41.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:19:41.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:19:41.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:19:41.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:19:41.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:19:41.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:19:41.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:19:41.362 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:19:41.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:41.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-17T06:19:41.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:42.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:19:42.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:42.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:19:42.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:19:42.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:42.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:42.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 == 0 2026-02-17T06:19:42.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:19:42.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:19:42.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:19:42.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:19:42.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:19:42.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:19:42.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:19:43.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=2 2026-02-17T06:19:43.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:19:43.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:43.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:43.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 2 = 2 2026-02-17T06:19:43.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:19:43.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:19:43.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 2 = 1 ']' 2026-02-17T06:19:43.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:19:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 2 = 6 ']' 2026-02-17T06:19:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:19:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-3 1 1 2026-02-17T06:19:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-3 1 1 2026-02-17T06:19:43.389 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-3' created 2026-02-17T06:19:43.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:19:44.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:19:44.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:19:44.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:19:44.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:19:44.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:19:44.402 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:19:44.402 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:19:44.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:19:44.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:19:44.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:19:44.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:19:44.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:19:44.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:19:44.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:19:44.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:19:44.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:19:44.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:19:44.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:19:44.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:19:44.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:19:44.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T06:19:44.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T06:19:44.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T06:19:44.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:19:44.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T06:19:44.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:19:44.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:19:44.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T06:19:44.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:19:44.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T06:19:44.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T06:19:44.744 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836486 2026-02-17T06:19:44.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:44.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-17T06:19:44.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:45.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:19:45.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:46.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836486 2026-02-17T06:19:46.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:47.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:19:47.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:47.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T06:19:47.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:19:47.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:47.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:47.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 == 0 2026-02-17T06:19:47.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:19:47.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:19:47.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:19:47.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:19:47.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:19:47.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:19:47.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:19:47.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T06:19:47.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:19:47.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:47.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:47.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 3 2026-02-17T06:19:47.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:19:47.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:19:47.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 3 = 1 ']' 2026-02-17T06:19:47.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:19:47.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 3 = 6 ']' 2026-02-17T06:19:47.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:19:47.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-4 1 1 2026-02-17T06:19:47.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-4 1 1 2026-02-17T06:19:47.963 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-4' created 2026-02-17T06:19:47.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:19:48.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:19:48.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:19:48.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:19:48.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:19:48.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:19:48.979 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:19:48.979 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:19:48.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:19:48.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:19:48.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:19:49.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:19:49.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:19:49.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:19:49.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:19:49.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:19:49.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:19:49.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:19:49.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:19:49.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:19:49.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:19:49.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T06:19:49.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T06:19:49.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T06:19:49.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:19:49.327 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T06:19:49.327 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:19:49.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:19:49.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T06:19:49.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:19:49.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T06:19:49.332 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T06:19:49.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T06:19:49.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:49.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T06:19:49.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:50.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:19:50.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:50.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-17T06:19:50.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:19:50.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:50.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:50.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:19:50.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:19:50.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:19:50.829 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:19:50.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:19:50.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:19:50.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:19:50.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:19:50.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:19:50.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:19:50.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:50.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:51.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:19:51.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:19:51.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:19:51.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 4 = 1 ']' 2026-02-17T06:19:51.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:19:51.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 4 = 6 ']' 2026-02-17T06:19:51.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:19:51.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-5 1 1 2026-02-17T06:19:51.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-5 1 1 2026-02-17T06:19:51.361 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-5' created 2026-02-17T06:19:51.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:19:52.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:19:52.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:19:52.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:19:52.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:19:52.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:19:52.374 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:19:52.375 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:19:52.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:19:52.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:19:52.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:19:52.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:19:52.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:19:52.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:19:52.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:19:52.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:19:52.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:19:52.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:19:52.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:19:52.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:19:52.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:19:52.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T06:19:52.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T06:19:52.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T06:19:52.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:19:52.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T06:19:52.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:19:52.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:19:52.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T06:19:52.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:19:52.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T06:19:52.723 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836489 2026-02-17T06:19:52.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T06:19:52.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:52.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T06:19:52.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:53.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:19:53.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:54.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T06:19:54.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:55.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:19:55.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:55.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T06:19:55.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:19:55.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:55.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:55.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T06:19:55.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:19:55.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:19:55.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:19:55.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:19:55.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:19:55.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:19:55.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:19:55.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T06:19:55.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:19:55.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:55.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:55.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T06:19:55.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:19:55.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:19:55.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 5 = 1 ']' 2026-02-17T06:19:55.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:19:55.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 5 = 6 ']' 2026-02-17T06:19:55.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:19:55.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-6 1 1 2026-02-17T06:19:55.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-6 1 1 2026-02-17T06:19:55.931 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-6' created 2026-02-17T06:19:55.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:19:56.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:19:56.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:19:56.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:19:56.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:19:56.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:19:56.945 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:19:56.945 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:19:56.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:19:56.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:19:56.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:19:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:19:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:19:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:19:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:19:57.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:19:57.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:19:57.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:19:57.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:19:57.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:19:57.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:19:57.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T06:19:57.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T06:19:57.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T06:19:57.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:19:57.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T06:19:57.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:19:57.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:19:57.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T06:19:57.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:19:57.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T06:19:57.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T06:19:57.294 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836491 2026-02-17T06:19:57.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T06:19:57.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:19:58.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:19:58.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:19:58.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T06:19:58.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:19:58.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:58.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:58.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T06:19:58.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:19:58.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:19:58.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:19:58.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:19:58.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:19:58.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:19:58.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:19:58.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T06:19:58.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:19:58.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:19:58.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:19:59.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T06:19:59.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:19:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:19:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 6 = 1 ']' 2026-02-17T06:19:59.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:19:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 6 = 6 ']' 2026-02-17T06:19:59.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5868: TEST_scrub_warning: ceph osd pool set psr_pool-6 deep_scrub_interval 1209600 2026-02-17T06:19:59.312 INFO:tasks.workunit.client.0.trial074.stderr:set pool 6 deep_scrub_interval to 1209600 2026-02-17T06:19:59.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:19:59.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-7 1 1 2026-02-17T06:19:59.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-7 1 1 2026-02-17T06:19:59.517 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-7' created 2026-02-17T06:19:59.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:20:00.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:20:00.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:20:00.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:20:00.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:20:00.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:20:00.531 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:20:00.532 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:20:00.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:20:00.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:20:00.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:20:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:20:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:20:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:20:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:20:00.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:20:00.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:20:00.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:20:00.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:20:00.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:00.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:20:00.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-17T06:20:00.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-17T06:20:00.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-17T06:20:00.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:00.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-17T06:20:00.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:00.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:20:00.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-17T06:20:00.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:00.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-17T06:20:00.886 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836493 2026-02-17T06:20:00.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-17T06:20:00.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:01.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-17T06:20:01.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:02.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:20:02.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:02.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-17T06:20:02.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:20:02.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:02.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:02.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 7 == 0 2026-02-17T06:20:02.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:20:02.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:20:02.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:20:02.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:20:02.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:20:02.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:20:02.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:20:02.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=7 2026-02-17T06:20:02.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:20:02.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:02.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:02.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 7 = 7 2026-02-17T06:20:02.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:20:02.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:20:02.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 7 = 1 ']' 2026-02-17T06:20:02.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:02.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 7 = 6 ']' 2026-02-17T06:20:02.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:02.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-8 1 1 2026-02-17T06:20:02.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-8 1 1 2026-02-17T06:20:02.908 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-8' created 2026-02-17T06:20:02.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:20:03.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:20:03.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:20:03.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:20:03.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:20:03.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:20:03.921 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:20:03.922 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:20:03.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:20:03.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:20:03.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:20:04.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:20:04.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:20:04.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:20:04.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:20:04.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:20:04.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:20:04.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:20:04.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:20:04.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:04.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:20:04.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836494 2026-02-17T06:20:04.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836494 2026-02-17T06:20:04.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494' 2026-02-17T06:20:04.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:04.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836494 2026-02-17T06:20:04.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:04.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:20:04.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836494 2026-02-17T06:20:04.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:04.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836494 2026-02-17T06:20:04.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-17T06:20:04.267 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836494 2026-02-17T06:20:04.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:04.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T06:20:04.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:05.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:20:05.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:05.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T06:20:05.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:06.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:20:06.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:06.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836494 2026-02-17T06:20:06.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:20:06.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:06.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:06.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 == 0 2026-02-17T06:20:06.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:20:06.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:20:06.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:20:06.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:20:06.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:20:06.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:20:06.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:20:07.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=8 2026-02-17T06:20:07.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:20:07.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:07.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:07.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 8 = 8 2026-02-17T06:20:07.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:20:07.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:20:07.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 8 = 1 ']' 2026-02-17T06:20:07.264 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:07.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 8 = 6 ']' 2026-02-17T06:20:07.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:07.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-9 1 1 2026-02-17T06:20:07.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-9 1 1 2026-02-17T06:20:07.466 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-9' created 2026-02-17T06:20:07.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:20:08.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:20:08.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:20:08.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:20:08.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:20:08.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:20:08.479 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:20:08.479 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:20:08.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:20:08.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:20:08.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:20:08.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:20:08.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:20:08.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:20:08.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:20:08.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:20:08.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:20:08.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:20:08.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:20:08.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:08.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:20:08.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836496 2026-02-17T06:20:08.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836496 2026-02-17T06:20:08.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496' 2026-02-17T06:20:08.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:08.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836496 2026-02-17T06:20:08.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:08.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:20:08.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836496 2026-02-17T06:20:08.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:08.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836496 2026-02-17T06:20:08.822 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836496 2026-02-17T06:20:08.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-17T06:20:08.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:08.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-17T06:20:08.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:09.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:20:09.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:10.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836496 2026-02-17T06:20:10.137 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:20:10.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:10.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:10.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 9 == 0 2026-02-17T06:20:10.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:20:10.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:20:10.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:20:10.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:20:10.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:20:10.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:20:10.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:20:10.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=9 2026-02-17T06:20:10.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:20:10.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:10.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:10.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 9 = 9 2026-02-17T06:20:10.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:20:10.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:20:10.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 9 = 1 ']' 2026-02-17T06:20:10.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:10.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 9 = 6 ']' 2026-02-17T06:20:10.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:10.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-10 1 1 2026-02-17T06:20:10.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-10 1 1 2026-02-17T06:20:10.831 INFO:tasks.workunit.client.0.trial074.stderr:pool 'psr_pool-10' created 2026-02-17T06:20:10.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:20:11.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T06:20:11.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:20:11.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:20:11.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:20:11.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:20:11.843 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:20:11.843 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:20:11.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:20:11.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:20:11.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:20:11.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:20:11.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:20:11.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:20:11.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:20:11.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:20:11.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:20:12.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:20:12.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:20:12.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:12.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:20:12.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T06:20:12.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T06:20:12.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T06:20:12.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:12.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T06:20:12.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:12.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:20:12.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T06:20:12.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:12.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T06:20:12.192 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836498 2026-02-17T06:20:12.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T06:20:12.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:12.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836498 2026-02-17T06:20:12.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:13.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:20:13.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:13.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836498 2026-02-17T06:20:13.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:14.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:20:14.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:14.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-17T06:20:14.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:20:14.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:14.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 10 == 0 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:20:14.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:20:14.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=10 2026-02-17T06:20:14.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:20:14.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:14.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:15.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 10 = 10 2026-02-17T06:20:15.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:20:15.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:20:15.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 10 = 1 ']' 2026-02-17T06:20:15.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:15.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 10 = 6 ']' 2026-02-17T06:20:15.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5873: TEST_scrub_warning: local primary=0 2026-02-17T06:20:15.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5875: TEST_scrub_warning: ceph osd set noscrub 2026-02-17T06:20:15.365 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:20:15.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5876: TEST_scrub_warning: ceph osd set nodeep-scrub 2026-02-17T06:20:15.570 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:20:15.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5877: TEST_scrub_warning: ceph config set global osd_scrub_interval_randomize_ratio 0 2026-02-17T06:20:15.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5878: TEST_scrub_warning: ceph config set global osd_deep_scrub_randomize_ratio 0 2026-02-17T06:20:15.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5879: TEST_scrub_warning: ceph config set global osd_scrub_max_interval 604800 2026-02-17T06:20:16.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5880: TEST_scrub_warning: ceph config set global osd_deep_scrub_interval 604800 2026-02-17T06:20:16.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: seq 1 5 2026-02-17T06:20:16.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T06:20:16.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 1 = 1 ']' 2026-02-17T06:20:16.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5887: TEST_scrub_warning: overdue_seconds=1900800 2026-02-17T06:20:16.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 1900800 + 100 2026-02-17T06:20:16.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 1.0 schedule-scrub 1900900 2026-02-17T06:20:16.303 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.303 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:20:16.303 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.303 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-01-26T06:16:56.299953+0000" 2026-02-17T06:20:16.303 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T06:20:16.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 2 = 1 ']' 2026-02-17T06:20:16.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 200 2026-02-17T06:20:16.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 2.0 schedule-scrub 993800 2026-02-17T06:20:16.375 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.376 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:20:16.376 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.376 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:15:16.372809+0000" 2026-02-17T06:20:16.376 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T06:20:16.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 3 = 1 ']' 2026-02-17T06:20:16.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 300 2026-02-17T06:20:16.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 3.0 schedule-scrub 993900 2026-02-17T06:20:16.451 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.451 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:20:16.451 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.452 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:13:36.448508+0000" 2026-02-17T06:20:16.452 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T06:20:16.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 4 = 1 ']' 2026-02-17T06:20:16.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 400 2026-02-17T06:20:16.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 4.0 schedule-scrub 994000 2026-02-17T06:20:16.527 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.527 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:20:16.527 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.527 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:11:56.524955+0000" 2026-02-17T06:20:16.527 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T06:20:16.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 5 = 1 ']' 2026-02-17T06:20:16.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 500 2026-02-17T06:20:16.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 5.0 schedule-scrub 994100 2026-02-17T06:20:16.602 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.602 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:20:16.602 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.602 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:10:16.599516+0000" 2026-02-17T06:20:16.602 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.610 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:16.612 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: expr 5 + 5 2026-02-17T06:20:16.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: seq 6 10 2026-02-17T06:20:16.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:16.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:16.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 6 = 6 ']' 2026-02-17T06:20:16.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5898: TEST_scrub_warning: overdue_seconds=1900800 2026-02-17T06:20:16.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 1900800 + 600 2026-02-17T06:20:16.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 6.0 schedule-deep-scrub 1901400 2026-02-17T06:20:16.683 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.683 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:20:16.683 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.683 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-01-26T06:08:36.680623+0000" 2026-02-17T06:20:16.683 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:16.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:16.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 7 = 6 ']' 2026-02-17T06:20:16.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 700 2026-02-17T06:20:16.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 7.0 schedule-deep-scrub 994300 2026-02-17T06:20:16.758 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.758 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:20:16.758 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.758 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:06:56.755026+0000" 2026-02-17T06:20:16.759 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:16.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 8 = 6 ']' 2026-02-17T06:20:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 800 2026-02-17T06:20:16.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 8.0 schedule-deep-scrub 994400 2026-02-17T06:20:16.832 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.832 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:20:16.832 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.832 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:05:16.829247+0000" 2026-02-17T06:20:16.833 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:16.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:16.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 9 = 6 ']' 2026-02-17T06:20:16.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 900 2026-02-17T06:20:16.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 9.0 schedule-deep-scrub 994500 2026-02-17T06:20:16.907 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.907 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:20:16.907 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.907 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:03:36.904390+0000" 2026-02-17T06:20:16.907 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T06:20:16.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T06:20:16.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 10 = 6 ']' 2026-02-17T06:20:16.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T06:20:16.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 1000 2026-02-17T06:20:16.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 10.0 schedule-deep-scrub 994600 2026-02-17T06:20:16.982 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:16.983 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:20:16.983 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:16.983 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-05T18:01:56.979729+0000" 2026-02-17T06:20:16.983 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:16.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5904: TEST_scrub_warning: flush_pg_stats 2026-02-17T06:20:16.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:20:16.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:20:17.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:20:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:20:17.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:17.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:20:17.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836500 2026-02-17T06:20:17.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836500 2026-02-17T06:20:17.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500' 2026-02-17T06:20:17.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:17.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836500 2026-02-17T06:20:17.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:17.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:20:17.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836500 2026-02-17T06:20:17.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:17.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836500 2026-02-17T06:20:17.217 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836500 2026-02-17T06:20:17.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-17T06:20:17.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:17.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836500 2026-02-17T06:20:17.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:18.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:20:18.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:18.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-17T06:20:18.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5906: TEST_scrub_warning: ceph health 2026-02-17T06:20:18.696 INFO:tasks.workunit.client.0.trial074.stdout:HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 5 pgs not deep-scrubbed in time; 10 pgs not scrubbed in time; 10 pool(s) do not have an application enabled; 10 pool(s) have no replicas configured; OSD count 1 < osd_pool_default_size 3 2026-02-17T06:20:18.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5907: TEST_scrub_warning: ceph health detail 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout:HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 5 pgs not deep-scrubbed in time; 10 pgs not scrubbed in time; 10 pool(s) do not have an application enabled; 10 pool(s) have no replicas configured; OSD count 1 < osd_pool_default_size 3 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout:[WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout:[WRN] PG_NOT_DEEP_SCRUBBED: 5 pgs not deep-scrubbed in time 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout: pg 10.0 not deep-scrubbed since 2026-02-05T18:01:56.979729+0000 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout: pg 9.0 not deep-scrubbed since 2026-02-05T18:03:36.904390+0000 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout: pg 8.0 not deep-scrubbed since 2026-02-05T18:05:16.829247+0000 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout: pg 7.0 not deep-scrubbed since 2026-02-05T18:06:56.755026+0000 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout: pg 6.0 not deep-scrubbed since 2026-01-26T06:08:36.680623+0000 2026-02-17T06:20:18.888 INFO:tasks.workunit.client.0.trial074.stdout:[WRN] PG_NOT_SCRUBBED: 10 pgs not scrubbed in time 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 10.0 not scrubbed since 2026-02-05T18:01:56.979729+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 9.0 not scrubbed since 2026-02-05T18:03:36.904390+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 8.0 not scrubbed since 2026-02-05T18:05:16.829247+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 7.0 not scrubbed since 2026-02-05T18:06:56.755026+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 6.0 not scrubbed since 2026-01-26T06:08:36.680623+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 5.0 not scrubbed since 2026-02-05T18:10:16.599516+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 4.0 not scrubbed since 2026-02-05T18:11:56.524955+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 3.0 not scrubbed since 2026-02-05T18:13:36.448508+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 2.0 not scrubbed since 2026-02-05T18:15:16.372809+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: pg 1.0 not scrubbed since 2026-01-26T06:16:56.299953+0000 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout:[WRN] POOL_APP_NOT_ENABLED: 10 pool(s) do not have an application enabled 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-1' 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-2' 2026-02-17T06:20:18.889 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-3' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-4' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-5' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-6' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-7' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-8' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-9' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: application not enabled on pool 'psr_pool-10' 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout:[WRN] POOL_NO_REDUNDANCY: 10 pool(s) have no replicas configured 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-1' has no replicas configured 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-2' has no replicas configured 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-3' has no replicas configured 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-4' has no replicas configured 2026-02-17T06:20:18.890 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-5' has no replicas configured 2026-02-17T06:20:18.891 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-6' has no replicas configured 2026-02-17T06:20:18.891 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-7' has no replicas configured 2026-02-17T06:20:18.891 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-8' has no replicas configured 2026-02-17T06:20:18.891 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-9' has no replicas configured 2026-02-17T06:20:18.891 INFO:tasks.workunit.client.0.trial074.stdout: pool 'psr_pool-10' has no replicas configured 2026-02-17T06:20:18.891 INFO:tasks.workunit.client.0.trial074.stdout:[WRN] TOO_FEW_OSDS: OSD count 1 < osd_pool_default_size 3 2026-02-17T06:20:18.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5908: TEST_scrub_warning: ceph health 2026-02-17T06:20:18.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5908: TEST_scrub_warning: grep -q ' pgs not deep-scrubbed in time' 2026-02-17T06:20:19.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5909: TEST_scrub_warning: ceph health 2026-02-17T06:20:19.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5909: TEST_scrub_warning: grep -q ' pgs not scrubbed in time' 2026-02-17T06:20:19.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: ceph health detail 2026-02-17T06:20:19.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: grep 'not scrubbed since' 2026-02-17T06:20:19.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: wc -l 2026-02-17T06:20:19.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: COUNT=10 2026-02-17T06:20:19.442 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5916: TEST_scrub_warning: expr 5+5 2026-02-17T06:20:19.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5916: TEST_scrub_warning: (( 10 != 5 && 10 != 5+5 )) 2026-02-17T06:20:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: ceph health detail 2026-02-17T06:20:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: grep 'not deep-scrubbed since' 2026-02-17T06:20:19.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: wc -l 2026-02-17T06:20:19.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: COUNT=5 2026-02-17T06:20:19.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5921: TEST_scrub_warning: '[' 5 '!=' 5 ']' 2026-02-17T06:20:19.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:20:19.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:20:19.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:20:19.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:20:19.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:20:19.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:20:19.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:20:19.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:20:19.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:20:19.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:20:19.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:20:19.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:20:19.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:20:19.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:20:19.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:20:19.743 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:20:19.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:19.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:20:19.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:20:19.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:19.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:20:19.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:20:19.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:20:19.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:20:19.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:19.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:19.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:20:19.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:20:19.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:20:19.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:20:19.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:20:19.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:20:19.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:20:19.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:20:19.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:20:19.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:20:19.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:20:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:20:19.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:20:19.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:20:19.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:19.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:20:19.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:20:19.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:19.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:20:19.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:20:19.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:20:19.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:20:19.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:19.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:19.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:20:19.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:20:19.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:20:19.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:20:19.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:20:19.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:19.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:19.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:20:19.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_skip_non_repair_during_recovery td/osd-scrub-repair 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:136: TEST_skip_non_repair_during_recovery: local dir=td/osd-scrub-repair 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:137: TEST_skip_non_repair_during_recovery: local poolname=rbd 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:139: TEST_skip_non_repair_during_recovery: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:20:19.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:20:19.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:20:19.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:20:19.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T06:20:19.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:20:19.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:19.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:19.801 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:19.801 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:19.801 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:19.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:19.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T06:20:19.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:20:19.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:20:19.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:20:19.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:20:19.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:20:19.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:20:19.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:20:19.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:20:19.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:20:19.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:20:19.890 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:20:19.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:20:19.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:140: TEST_skip_non_repair_during_recovery: run_mgr td/osd-scrub-repair x 2026-02-17T06:20:19.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:20:19.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:20:19.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:20:19.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:20:19.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:20:19.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:20:20.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:20:20.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:20.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:20.042 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:20.042 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:20.042 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:20.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:20.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:20:20.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:20:20.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:141: TEST_skip_non_repair_during_recovery: run_osd td/osd-scrub-repair 0 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:20:20.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:20.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:20.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:20.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:20:20.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2f85df21-47e4-47ea-83ce-6809354ea074 2026-02-17T06:20:20.070 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 2f85df21-47e4-47ea-83ce-6809354ea074 2026-02-17T06:20:20.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2f85df21-47e4-47ea-83ce-6809354ea074' 2026-02-17T06:20:20.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:20:20.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCkCJRpuaKOBBAAsV2gUbnRoRiQKdpEKAQ/pw== 2026-02-17T06:20:20.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCkCJRpuaKOBBAAsV2gUbnRoRiQKdpEKAQ/pw=="}' 2026-02-17T06:20:20.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2f85df21-47e4-47ea-83ce-6809354ea074 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:20:20.168 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:20:20.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:20:20.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQCkCJRpuaKOBBAAsV2gUbnRoRiQKdpEKAQ/pw== --osd-uuid 2f85df21-47e4-47ea-83ce-6809354ea074 2026-02-17T06:20:20.194 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:20.186+0000 7f6b3f90c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:20.195 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:20.186+0000 7f6b3f90c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:20.196 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:20.190+0000 7f6b3f90c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:20.196 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:20.190+0000 7f6b3f90c8c0 -1 bdev(0x555f52287c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:20:20.196 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:20.190+0000 7f6b3f90c8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:20:23.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:20:23.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:20:23.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:20:23.238 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:20:23.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:20:23.432 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:20:23.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:20:23.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T06:20:23.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:20:23.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:20:23.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:20:23.456 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:23.450+0000 7f02e3afa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:23.458 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:23.450+0000 7f02e3afa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:23.459 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:23.450+0000 7f02e3afa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:23.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:20:23.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:20:23.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:20:23.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:20:23.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:20:23.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:23.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:20:23.582 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:20:23.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:23.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:23.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:24.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:24.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:24.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:20:24.738 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:20:24.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:24.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:24.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:24.933 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:24.926+0000 7f02e3afa8c0 -1 Falling back to public interface 2026-02-17T06:20:25.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:25.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:25.895 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:20:25.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:20:25.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:25.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:26.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:27.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:27.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:27.047 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:20:27.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:20:27.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:27.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:27.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:27.929 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:27.922+0000 7f02e3afa8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:20:28.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:28.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:28.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:20:28.203 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:20:28.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:28.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:28.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:29.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:29.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:29.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:20:29.355 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:20:29.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:29.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:29.509 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/991310343,v1:127.0.0.1:6803/991310343] [v2:127.0.0.1:6804/991310343,v1:127.0.0.1:6805/991310343] exists,up 2f85df21-47e4-47ea-83ce-6809354ea074 2026-02-17T06:20:29.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:20:29.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:20:29.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:20:29.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:144: TEST_skip_non_repair_during_recovery: run_osd td/osd-scrub-repair 1 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:29.510 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:20:29.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T06:20:29.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:20:29.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:20:29.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=be73e7e0-d4d9-41e8-9557-ca5da3340199 2026-02-17T06:20:29.514 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 be73e7e0-d4d9-41e8-9557-ca5da3340199 2026-02-17T06:20:29.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 be73e7e0-d4d9-41e8-9557-ca5da3340199' 2026-02-17T06:20:29.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:20:29.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCtCJRprFw1HxAAlVzsTFfWqe1q0wEd8lyb1w== 2026-02-17T06:20:29.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCtCJRprFw1HxAAlVzsTFfWqe1q0wEd8lyb1w=="}' 2026-02-17T06:20:29.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new be73e7e0-d4d9-41e8-9557-ca5da3340199 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:20:29.685 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:20:29.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:20:29.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQCtCJRprFw1HxAAlVzsTFfWqe1q0wEd8lyb1w== --osd-uuid be73e7e0-d4d9-41e8-9557-ca5da3340199 2026-02-17T06:20:29.712 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:29.706+0000 7f7da32ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:29.714 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:29.706+0000 7f7da32ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:29.715 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:29.706+0000 7f7da32ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:29.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:29.710+0000 7f7da32ad8c0 -1 bdev(0x5642d96a5c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:20:29.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:29.710+0000 7f7da32ad8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:20:31.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:20:31.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:20:31.985 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:20:31.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:20:31.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:20:32.173 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:20:32.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:20:32.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T06:20:32.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:20:32.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:20:32.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:20:32.197 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:32.190+0000 7fb25d5928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:32.199 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:32.190+0000 7fb25d5928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:32.200 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:32.194+0000 7fb25d5928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:20:32.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:33.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:33.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:33.476 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:20:33.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:20:33.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:33.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:20:33.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:33.697 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:33.690+0000 7fb25d5928c0 -1 Falling back to public interface 2026-02-17T06:20:34.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:34.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:34.634 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:20:34.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:20:34.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:34.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:20:34.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:35.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:35.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:35.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:20:35.784 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:20:35.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:35.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:20:35.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:36.723 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:36.714+0000 7fb25d5928c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:20:36.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:36.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:36.939 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:20:36.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:20:36.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:36.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:20:37.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:38.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:38.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:38.113 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:20:38.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:20:38.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:38.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:20:38.262 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2519054087,v1:127.0.0.1:6811/2519054087] [v2:127.0.0.1:6812/2519054087,v1:127.0.0.1:6813/2519054087] exists,up be73e7e0-d4d9-41e8-9557-ca5da3340199 2026-02-17T06:20:38.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:20:38.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:20:38.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:20:38.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:147: TEST_skip_non_repair_during_recovery: create_rbd_pool 2026-02-17T06:20:38.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T06:20:38.404 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' does not exist 2026-02-17T06:20:38.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T06:20:38.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T06:20:38.626 INFO:tasks.workunit.client.0.trial074.stderr:pool 'rbd' created 2026-02-17T06:20:38.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:20:39.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T06:20:39.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:148: TEST_skip_non_repair_during_recovery: wait_for_clean 2026-02-17T06:20:39.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:20:39.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:20:39.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:20:39.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:20:39.919 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:20:39.919 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:20:39.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:20:39.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:20:39.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:20:40.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:20:40.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:20:40.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:20:40.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:20:40.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:20:40.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:20:40.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:20:40.185 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:20:40.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:20:40.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:40.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:20:40.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:20:40.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:20:40.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:20:40.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:40.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:20:40.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T06:20:40.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T06:20:40.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T06:20:40.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:40.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:20:40.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:40.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:20:40.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:20:40.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:40.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:20:40.335 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:20:40.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:20:40.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:40.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:20:40.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:40.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T06:20:40.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:40.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:20:40.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T06:20:40.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:40.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T06:20:40.500 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705666 2026-02-17T06:20:40.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T06:20:40.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:20:40.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705665 -lt 38654705666 2026-02-17T06:20:40.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:41.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:20:41.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:20:41.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705665 -lt 38654705666 2026-02-17T06:20:41.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:42.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:20:42.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:20:42.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T06:20:42.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:20:42.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:42.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:43.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:20:43.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:20:43.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:20:43.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:20:43.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:20:43.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:20:43.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:20:43.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:20:43.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:20:43.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:20:43.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:43.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:43.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:20:43.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:150: TEST_skip_non_repair_during_recovery: add_something td/osd-scrub-repair rbd 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:20:43.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:20:43.669 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:20:43.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:20:43.875 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:20:43.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:20:43.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:20:43.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T06:20:43.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:151: TEST_skip_non_repair_during_recovery: get_not_primary rbd SOMETHING 2026-02-17T06:20:43.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=rbd 2026-02-17T06:20:43.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T06:20:43.913 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary rbd SOMETHING 2026-02-17T06:20:43.913 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T06:20:43.913 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T06:20:43.914 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T06:20:43.914 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:20:44.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T06:20:44.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T06:20:44.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T06:20:44.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:151: TEST_skip_non_repair_during_recovery: scrub_and_not_schedule td/osd-scrub-repair rbd 0 2026-02-17T06:20:44.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:155: scrub_and_not_schedule: local dir=td/osd-scrub-repair 2026-02-17T06:20:44.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:156: scrub_and_not_schedule: local poolname=rbd 2026-02-17T06:20:44.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:157: scrub_and_not_schedule: local osd=0 2026-02-17T06:20:44.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:162: scrub_and_not_schedule: get_pg rbd SOMETHING 2026-02-17T06:20:44.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T06:20:44.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T06:20:44.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T06:20:44.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:20:44.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:162: scrub_and_not_schedule: local pg=1.3 2026-02-17T06:20:44.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:163: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T06:20:44.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T06:20:44.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:20:44.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:20:44.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T06:20:44.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:163: scrub_and_not_schedule: local last_scrub=2026-02-17T06:20:38.622989+0000 2026-02-17T06:20:44.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:164: scrub_and_not_schedule: ceph tell 1.3 schedule-scrub 2026-02-17T06:20:44.586 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:20:44.586 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:20:44.587 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:20:44.587 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:19:04.584015+0000" 2026-02-17T06:20:44.587 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:20:44.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i=0 )) 2026-02-17T06:20:44.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T06:20:44.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T06:20:44.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T06:20:44.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:20:44.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:20:44.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T06:20:44.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: test 2026-02-17T06:20:38.622989+0000 '>' 2026-02-17T06:20:38.622989+0000 2026-02-17T06:20:44.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:173: scrub_and_not_schedule: sleep 1 2026-02-17T06:20:45.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i++ )) 2026-02-17T06:20:45.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T06:20:45.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T06:20:45.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T06:20:45.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:20:45.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T06:20:45.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:20:45.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: test 2026-02-17T06:20:38.622989+0000 '>' 2026-02-17T06:20:38.622989+0000 2026-02-17T06:20:45.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:173: scrub_and_not_schedule: sleep 1 2026-02-17T06:20:46.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i++ )) 2026-02-17T06:20:46.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T06:20:46.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T06:20:46.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T06:20:46.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:20:46.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:20:46.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T06:20:47.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: test 2026-02-17T06:20:38.622989+0000 '>' 2026-02-17T06:20:38.622989+0000 2026-02-17T06:20:47.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:173: scrub_and_not_schedule: sleep 1 2026-02-17T06:20:48.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i++ )) 2026-02-17T06:20:48.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T06:20:48.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:179: scrub_and_not_schedule: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:20:48.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:20:48.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:20:48.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:20:48.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:20:48.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:20:49.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:20:49.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T06:20:49.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:20:49.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:20:49.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:20:49.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:20:49.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:20:49.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T06:20:49.687 INFO:tasks.workunit.client.0.trial074.stdout:_ 2026-02-17T06:20:49.688 INFO:tasks.workunit.client.0.trial074.stdout:snapset 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:20:50.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:50.228 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:20:50.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:20:50.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:20:50.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:20:50.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:20:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:20:50.231 INFO:tasks.workunit.client.0.trial074.stderr:start osd.0 2026-02-17T06:20:50.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:20:50.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:20:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:20:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:20:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:20:50.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:20:50.243 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:50.238+0000 7ff5fdcd18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:50.245 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:50.238+0000 7ff5fdcd18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:50.246 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:50.238+0000 7ff5fdcd18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:50.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:20:50.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:50.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:50.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:51.481 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:51.474+0000 7ff5fdcd18c0 -1 Falling back to public interface 2026-02-17T06:20:51.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:51.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:51.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:20:51.538 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:20:51.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:51.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:51.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:52.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:52.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:52.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:20:52.689 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:20:52.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:52.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:52.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:53.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:53.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:53.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:20:53.841 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:20:53.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:53.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:53.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:20:54.486 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:54.478+0000 7ff5fdcd18c0 -1 osd.0 18 log_to_monitors true 2026-02-17T06:20:54.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:20:54.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:20:54.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:20:54.993 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:20:54.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:20:54.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:20:55.143 INFO:tasks.workunit.client.0.trial074.stderr:osd.0 up in weight 1 up_from 22 up_thru 22 down_at 19 last_clean_interval [5,18) [v2:127.0.0.1:6802/2457477928,v1:127.0.0.1:6803/2457477928] [v2:127.0.0.1:6804/2457477928,v1:127.0.0.1:6805/2457477928] exists,up 2f85df21-47e4-47ea-83ce-6809354ea074 2026-02-17T06:20:55.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:20:55.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:20:55.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:20:55.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:20:55.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:20:55.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:20:55.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:20:55.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:20:55.144 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:20:55.144 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:20:55.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:20:55.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:20:55.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:20:55.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:20:55.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:20:55.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:20:55.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:20:55.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:20:55.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:20:55.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:20:55.410 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:20:55.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:20:55.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:55.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:20:55.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280514 2026-02-17T06:20:55.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280514 2026-02-17T06:20:55.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280514' 2026-02-17T06:20:55.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:20:55.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:20:55.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T06:20:55.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T06:20:55.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-94489280514 1-38654705670' 2026-02-17T06:20:55.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:55.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-94489280514 2026-02-17T06:20:55.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:55.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:20:55.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-94489280514 2026-02-17T06:20:55.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:55.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280514 2026-02-17T06:20:55.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 94489280514' 2026-02-17T06:20:55.558 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 94489280514 2026-02-17T06:20:55.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:55.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 94489280514 2026-02-17T06:20:55.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:20:56.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:20:56.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:20:56.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280514 -lt 94489280514 2026-02-17T06:20:56.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:20:56.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T06:20:56.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:20:56.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:20:56.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T06:20:56.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:20:56.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T06:20:56.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T06:20:56.876 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705670 2026-02-17T06:20:56.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:20:57.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T06:20:57.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:20:57.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:57.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:57.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T06:20:57.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:20:57.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:20:57.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:20:57.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:20:57.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:20:57.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:20:57.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:20:57.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T06:20:57.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:20:57.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:20:57.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:20:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T06:20:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:20:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:20:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:180: scrub_and_not_schedule: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T06:20:57.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:181: scrub_and_not_schedule: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T06:20:57.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:20:57.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:20:57.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:20:57.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:20:57.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:20:57.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:20:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:20:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:20:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:20:57.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:20:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:20:57.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:20:57.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:20:57.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:20:57.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:20:57.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:20:57.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:57.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:20:57.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:20:57.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:57.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:20:57.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:20:57.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:20:57.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:20:57.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:20:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:20:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:20:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:20:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:20:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:20:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:20:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:20:57.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:20:57.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:20:57.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:20:57.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:20:57.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:20:57.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:20:57.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:20:57.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:20:57.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:20:57.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:20:57.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:20:57.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:20:57.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:20:57.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:20:57.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:57.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:20:57.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:20:57.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:20:57.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:20:57.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:20:57.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:20:57.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:20:57.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:20:57.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:20:57.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:20:57.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:20:57.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:20:57.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:20:57.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:20:57.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_unfound_erasure_coded_appends td/osd-scrub-repair 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:903: TEST_unfound_erasure_coded_appends: unfound_erasure_coded td/osd-scrub-repair false 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:852: unfound_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:853: unfound_erasure_coded: local allow_overwrites=false 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:854: unfound_erasure_coded: local poolname=ecpool 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:855: unfound_erasure_coded: local payload=ABCDEF 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:857: unfound_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T06:20:57.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:20:57.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:20:57.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:20:57.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:20:57.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:20:57.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:20:57.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:20:57.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:57.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:57.730 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:57.731 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.731 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:57.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:20:57.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:20:57.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:20:57.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:20:57.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:20:57.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:20:57.759 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:20:57.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:20:57.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:20:57.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:20:57.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:20:57.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:20:57.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:20:57.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:20:57.818 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:20:57.818 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:20:57.818 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:20:57.818 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:20:57.818 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.818 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.818 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:20:57.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:20:57.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:20:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:858: unfound_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T06:20:57.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:20:57.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:20:57.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:20:57.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:20:57.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:20:57.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:20:57.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:20:57.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:57.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:57.969 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:57.969 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.969 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:57.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:20:57.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:20:57.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: seq 0 3 2026-02-17T06:20:57.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:20:57.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:20:57.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:20:57.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:20:57.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:20:57.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:20:57.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:20:57.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=09a997e7-8bfa-4d21-bb95-8853f58aa814 2026-02-17T06:20:57.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 09a997e7-8bfa-4d21-bb95-8853f58aa814' 2026-02-17T06:20:57.996 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 09a997e7-8bfa-4d21-bb95-8853f58aa814 2026-02-17T06:20:57.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:20:58.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDKCJRpyJsXABAAlRDe+AgwhZM50QdTdJdNow== 2026-02-17T06:20:58.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDKCJRpyJsXABAAlRDe+AgwhZM50QdTdJdNow=="}' 2026-02-17T06:20:58.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 09a997e7-8bfa-4d21-bb95-8853f58aa814 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:20:58.094 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:20:58.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:20:58.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDKCJRpyJsXABAAlRDe+AgwhZM50QdTdJdNow== --osd-uuid 09a997e7-8bfa-4d21-bb95-8853f58aa814 2026-02-17T06:20:58.118 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:58.110+0000 7fe9e88c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:58.120 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:58.114+0000 7fe9e88c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:58.121 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:58.114+0000 7fe9e88c78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:20:58.121 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:58.114+0000 7fe9e88c78c0 -1 bdev(0x558b46b57c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:20:58.122 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:20:58.114+0000 7fe9e88c78c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:21:00.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:21:00.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:21:00.714 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:21:00.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:21:00.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:21:00.904 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:21:00.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:21:00.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:21:00.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:21:00.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:21:00.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:21:00.927 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:00.918+0000 7fcedd9f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:00.928 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:00.922+0000 7fcedd9f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:00.929 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:00.922+0000 7fcedd9f08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:01.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:21:01.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:21:01.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:21:01.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:21:01.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:21:01.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:01.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:21:01.058 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:21:01.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:01.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:01.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:02.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:21:02.210 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:21:02.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:02.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:02.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:02.417 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:02.410+0000 7fcedd9f08c0 -1 Falling back to public interface 2026-02-17T06:21:03.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:03.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:03.360 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:21:03.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:21:03.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:03.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:03.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:04.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:04.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:04.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:21:04.510 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:21:04.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:04.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:04.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:05.388 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:05.382+0000 7fcedd9f08c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:21:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:05.663 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:21:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:21:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:05.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:05.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:06.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:06.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:06.819 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:21:06.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:21:06.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:06.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:06.966 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1589300062,v1:127.0.0.1:6803/1589300062] [v2:127.0.0.1:6804/1589300062,v1:127.0.0.1:6805/1589300062] exists,up 09a997e7-8bfa-4d21-bb95-8853f58aa814 2026-02-17T06:21:06.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:21:06.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:21:06.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:06.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:06.968 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:06.968 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:06.968 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:06.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:06.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:21:06.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:21:06.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:21:06.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=743945db-2467-49af-87e8-11e3a7a994db 2026-02-17T06:21:06.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 743945db-2467-49af-87e8-11e3a7a994db' 2026-02-17T06:21:06.971 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 743945db-2467-49af-87e8-11e3a7a994db 2026-02-17T06:21:06.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:21:06.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDSCJRpZ5J8OhAAezdQlalu/z31wIdtfc5lag== 2026-02-17T06:21:06.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDSCJRpZ5J8OhAAezdQlalu/z31wIdtfc5lag=="}' 2026-02-17T06:21:06.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 743945db-2467-49af-87e8-11e3a7a994db -i td/osd-scrub-repair/1/new.json 2026-02-17T06:21:07.135 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:21:07.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:21:07.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDSCJRpZ5J8OhAAezdQlalu/z31wIdtfc5lag== --osd-uuid 743945db-2467-49af-87e8-11e3a7a994db 2026-02-17T06:21:07.161 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:07.154+0000 7f1986cd48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:07.162 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:07.154+0000 7f1986cd48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:07.163 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:07.158+0000 7f1986cd48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:07.164 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:07.158+0000 7f1986cd48c0 -1 bdev(0x56545e8d3c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:21:07.164 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:07.158+0000 7f1986cd48c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:21:09.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:21:09.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:21:09.454 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:21:09.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:21:09.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:21:09.649 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:21:09.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:21:09.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:21:09.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:21:09.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:21:09.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:21:09.664 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:09.658+0000 7fa9366f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:09.665 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:09.658+0000 7fa9366f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:09.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:09.658+0000 7fa9366f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:09.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:21:09.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:21:09.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:21:09.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:21:09.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:21:09.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:09.802 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:21:09.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:21:09.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:09.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:21:09.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:10.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:10.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:10.955 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:21:10.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:21:10.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:10.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:21:11.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:11.409 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:11.402+0000 7fa9366f38c0 -1 Falling back to public interface 2026-02-17T06:21:12.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:12.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:12.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:21:12.107 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:21:12.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:12.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:21:12.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:13.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:13.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:13.258 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:21:13.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:21:13.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:13.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:21:13.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:14.382 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:14.374+0000 7fa9366f38c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:21:14.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:14.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:14.411 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:21:14.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:21:14.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:14.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:21:14.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:15.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:15.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:15.561 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:21:15.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:21:15.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:15.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:21:15.708 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4078345570,v1:127.0.0.1:6811/4078345570] [v2:127.0.0.1:6812/4078345570,v1:127.0.0.1:6813/4078345570] exists,up 743945db-2467-49af-87e8-11e3a7a994db 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:21:15.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:15.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:21:15.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:21:15.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:21:15.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=60846cb7-a32b-4cd3-8964-55413f0355c6 2026-02-17T06:21:15.714 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 60846cb7-a32b-4cd3-8964-55413f0355c6 2026-02-17T06:21:15.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 60846cb7-a32b-4cd3-8964-55413f0355c6' 2026-02-17T06:21:15.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:21:15.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDbCJRpK6IaKxAAMmed4Q3ZFGmUA/vNpFB1xw== 2026-02-17T06:21:15.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDbCJRpK6IaKxAAMmed4Q3ZFGmUA/vNpFB1xw=="}' 2026-02-17T06:21:15.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 60846cb7-a32b-4cd3-8964-55413f0355c6 -i td/osd-scrub-repair/2/new.json 2026-02-17T06:21:15.882 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:21:15.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T06:21:15.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDbCJRpK6IaKxAAMmed4Q3ZFGmUA/vNpFB1xw== --osd-uuid 60846cb7-a32b-4cd3-8964-55413f0355c6 2026-02-17T06:21:15.916 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:15.910+0000 7f4ec32ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:15.917 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:15.910+0000 7f4ec32ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:15.919 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:15.910+0000 7f4ec32ab8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:15.919 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:15.914+0000 7f4ec32ab8c0 -1 bdev(0x562208fedc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:21:15.919 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:15.914+0000 7f4ec32ab8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T06:21:18.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T06:21:18.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:21:18.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:21:18.255 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:21:18.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:21:18.443 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:21:18.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:21:18.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:21:18.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:21:18.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:21:18.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:21:18.468 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:18.462+0000 7fd2b122a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:18.469 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:18.462+0000 7fd2b122a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:18.470 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:18.462+0000 7fd2b122a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:18.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:21:18.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:21:18.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:21:18.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:21:18.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:21:18.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:18.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:21:18.601 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:21:18.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:18.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:18.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:19.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:19.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:19.754 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:21:19.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:21:19.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:19.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:19.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:19.981 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:19.973+0000 7fd2b122a8c0 -1 Falling back to public interface 2026-02-17T06:21:20.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:20.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:20.904 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:21:20.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:21:20.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:20.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:21.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:22.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:22.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:22.054 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:21:22.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:21:22.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:22.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:22.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:22.956 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:22.949+0000 7fd2b122a8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:21:23.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:23.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:23.206 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:21:23.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:21:23.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:23.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:23.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:24.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:24.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:24.360 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:21:24.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:21:24.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:24.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:24.510 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2121130405,v1:127.0.0.1:6819/2121130405] [v2:127.0.0.1:6820/2121130405,v1:127.0.0.1:6821/2121130405] exists,up 60846cb7-a32b-4cd3-8964-55413f0355c6 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 3 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:21:24.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:24.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:24.513 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:24.513 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:24.513 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:24.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:24.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T06:21:24.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:21:24.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8e61bc24-46a3-4992-94de-4a83ea303a8a 2026-02-17T06:21:24.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 8e61bc24-46a3-4992-94de-4a83ea303a8a' 2026-02-17T06:21:24.515 INFO:tasks.workunit.client.0.trial074.stdout:add osd3 8e61bc24-46a3-4992-94de-4a83ea303a8a 2026-02-17T06:21:24.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:21:24.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDkCJRpbQxOHxAAxg4dBe/fsbHpnfTgjsTfqg== 2026-02-17T06:21:24.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDkCJRpbQxOHxAAxg4dBe/fsbHpnfTgjsTfqg=="}' 2026-02-17T06:21:24.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8e61bc24-46a3-4992-94de-4a83ea303a8a -i td/osd-scrub-repair/3/new.json 2026-02-17T06:21:24.686 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:21:24.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T06:21:24.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDkCJRpbQxOHxAAxg4dBe/fsbHpnfTgjsTfqg== --osd-uuid 8e61bc24-46a3-4992-94de-4a83ea303a8a 2026-02-17T06:21:24.711 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:24.705+0000 7fbd460f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:24.713 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:24.705+0000 7fbd460f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:24.714 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:24.705+0000 7fbd460f58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:24.715 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:24.705+0000 7fbd460f58c0 -1 bdev(0x55d95f319c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T06:21:24.715 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:24.705+0000 7fbd460f58c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T06:21:26.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T06:21:26.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:21:26.993 INFO:tasks.workunit.client.0.trial074.stdout:adding osd3 key to auth repository 2026-02-17T06:21:26.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T06:21:26.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:21:27.186 INFO:tasks.workunit.client.0.trial074.stdout:start osd.3 2026-02-17T06:21:27.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T06:21:27.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:21:27.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:21:27.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:21:27.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:21:27.210 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:27.201+0000 7f70e4fb78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:27.212 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:27.205+0000 7f70e4fb78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:27.213 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:27.205+0000 7f70e4fb78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:27.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T06:21:27.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:21:27.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T06:21:27.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:21:27.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:21:27.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:27.339 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:21:27.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:21:27.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:27.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:27.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:28.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:28.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:28.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:21:28.491 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:21:28.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:28.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:28.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:28.733 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:28.725+0000 7f70e4fb78c0 -1 Falling back to public interface 2026-02-17T06:21:29.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:29.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:29.638 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:21:29.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:21:29.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:29.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:29.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:30.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:30.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:30.789 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:21:30.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:21:30.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:30.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:30.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:31.734 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:31.725+0000 7f70e4fb78c0 -1 osd.3 0 log_to_monitors true 2026-02-17T06:21:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:31.936 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:21:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:21:31.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:31.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:32.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:33.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:33.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:33.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:21:33.093 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:21:33.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:33.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:33.241 INFO:tasks.workunit.client.0.trial074.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/571641765,v1:127.0.0.1:6827/571641765] [v2:127.0.0.1:6828/571641765,v1:127.0.0.1:6829/571641765] exists,up 8e61bc24-46a3-4992-94de-4a83ea303a8a 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: unfound_erasure_coded: create_ec_pool ecpool false k=2 m=2 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T06:21:33.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T06:21:33.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T06:21:33.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T06:21:33.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T06:21:33.740 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T06:21:33.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:21:34.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T06:21:34.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T06:21:34.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:21:34.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:21:34.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:21:34.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:21:34.753 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:21:34.753 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:21:34.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:21:34.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:21:34.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:21:34.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:21:34.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:21:34.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:21:34.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:21:34.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:21:34.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:21:35.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:21:35.022 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:21:35.022 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:21:35.022 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T06:21:35.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:21:35.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:35.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:21:35.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T06:21:35.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T06:21:35.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T06:21:35.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:35.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:21:35.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672965 2026-02-17T06:21:35.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672965 2026-02-17T06:21:35.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2026-02-17T06:21:35.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:35.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:21:35.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T06:21:35.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T06:21:35.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509444' 2026-02-17T06:21:35.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:35.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:21:35.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378626 2026-02-17T06:21:35.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378626 2026-02-17T06:21:35.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-64424509444 3-81604378626' 2026-02-17T06:21:35.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:35.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T06:21:35.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:35.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:21:35.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T06:21:35.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:35.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T06:21:35.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T06:21:35.314 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T06:21:35.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:35.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T06:21:35.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:21:36.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:21:36.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:36.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T06:21:36.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:36.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672965 2026-02-17T06:21:36.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:36.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:21:36.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672965 2026-02-17T06:21:36.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:36.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672965 2026-02-17T06:21:36.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-17T06:21:36.627 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672965 2026-02-17T06:21:36.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:21:36.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672965 2026-02-17T06:21:36.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:36.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T06:21:36.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:36.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:21:36.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T06:21:36.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:36.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T06:21:36.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T06:21:36.790 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509444 2026-02-17T06:21:36.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:21:36.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T06:21:36.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:36.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-81604378626 2026-02-17T06:21:36.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:36.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:21:36.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-81604378626 2026-02-17T06:21:36.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:36.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378626 2026-02-17T06:21:36.947 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 81604378626 2026-02-17T06:21:36.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 81604378626' 2026-02-17T06:21:36.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:21:37.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378626 -lt 81604378626 2026-02-17T06:21:37.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:21:37.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:37.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:37.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:21:37.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:21:37.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:21:37.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:21:37.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:21:37.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:21:37.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:21:37.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:21:37.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:21:37.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:21:37.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:37.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:37.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:21:37.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:21:37.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:21:37.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T06:21:37.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:865: unfound_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T06:21:37.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:21:37.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:21:37.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T06:21:37.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:21:37.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:21:37.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:21:37.810 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:21:37.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:21:38.016 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:21:38.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:21:38.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:21:38.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T06:21:38.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T06:21:38.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T06:21:38.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T06:21:38.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T06:21:38.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:21:38.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: local primary=1 2026-02-17T06:21:38.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T06:21:38.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: sed -e s/1// 2026-02-17T06:21:38.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T06:21:38.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T06:21:38.211 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:21:38.211 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T06:21:38.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:21:38.364 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:21:38.364 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:21:38.364 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T06:21:38.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2 3 2026-02-17T06:21:38.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: osds=('0' '2' '3') 2026-02-17T06:21:38.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: local -a osds 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:869: unfound_erasure_coded: local not_primary_first=0 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:870: unfound_erasure_coded: local not_primary_second=2 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:871: unfound_erasure_coded: local not_primary_third=3 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:876: unfound_erasure_coded: pids= 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:877: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 435727"' 2026-02-17T06:21:38.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 435727' 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:878: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 435728"' 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 435728' 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:879: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:21:38.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 435732"' 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/435730: /' 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 435732' 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:880: unfound_erasure_coded: wait_background pids 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 435727 435728 435732' 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 435727 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/435733: /' 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:21:38.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/435736: /' 2026-02-17T06:21:40.881 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:21:40.882 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:21:40.883 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: remove 2#1:eb822e21:::SOMETHING:head# 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:21:40.884 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:21:40.885 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:21:40.885 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:40.886 INFO:tasks.workunit.client.0.trial074.stderr:435733: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:21:40.887 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:21:40.888 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:21:40.888 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:21:40.888 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:21:40.888 INFO:tasks.workunit.client.0.trial074.stderr:435733: start osd.2 2026-02-17T06:21:40.888 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: remove 3#1:eb822e21:::SOMETHING:head# 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.120 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:21:41.121 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T06:21:41.121 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:21:41.121 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T06:21:41.121 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:21:41.121 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:21:41.123 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:21:41.123 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:21:41.123 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T06:21:41.123 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T06:21:41.123 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: start osd.3 2026-02-17T06:21:41.124 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:21:41.138 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.138 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: remove 1#1:eb822e21:::SOMETHING:head# 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:21:41.139 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:21:41.140 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:21:41.140 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:21:41.140 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:21:41.140 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:21:41.140 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:21:41.140 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:41.142 INFO:tasks.workunit.client.0.trial074.stderr:435730: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: start osd.0 2026-02-17T06:21:41.143 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 2026-02-17T06:21:41.129+0000 7ff475cf08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 2026-02-17T06:21:41.133+0000 7ff475cf08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 2026-02-17T06:21:41.133+0000 7ff475cf08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 0 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 1 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 2026-02-17T06:21:42.485+0000 7ff475cf08c0 -1 Falling back to public interface 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 2 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: 3 2026-02-17T06:21:45.527 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.528 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:45.528 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.648 INFO:tasks.workunit.client.0.trial074.stderr:435736: 2026-02-17T06:21:45.517+0000 7ff475cf08c0 435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:21:45.648 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:21:45.648 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:21:45.648 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: 2026-02-17T06:21:40.897+0000 7fdb4c57a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: 2026-02-17T06:21:40.897+0000 7fdb4c57a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: 2026-02-17T06:21:40.897+0000 7fdb4c57a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:21:45.649 INFO:tasks.workunit.client.0.trial074.stderr:435733: 0 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: 1 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: 2026-02-17T06:21:42.665+0000 7fdb4c57a8c0 -1 Falling back to public interface 2026-02-17T06:21:45.650 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: 2 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: 3 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:45.651 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.663 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/st435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:21:45.663 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:21:45.663 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:21:45.663 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: 2026-02-17T06:21:41.157+0000 7fe0cda4d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: 2026-02-17T06:21:41.157+0000 7fe0cda4d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: 2026-02-17T06:21:41.161+0000 7fe0cda4d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.664 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: 0 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: 1 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: 2026-02-17T06:21:42.509+0000 7fe0cda4d8c0 -1 Falling back to public interface 2026-02-17T06:21:45.665 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: 2 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: 3 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:45.666 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:45.667 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: 2026-02-17T06:21:45.653+0000 7fe0cda4d8c0 -1 osd.0 26 log_to_monitors true 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: 4 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:21:46.391 INFO:tasks.workunit.client.0.trial074.stderr:435730: osd.0 up in weight 1 up_from 31 up_thru 0 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6818/4102619954,v1:127.0.0.1:6819/4102619954] [v2:127.0.0.1:6820/4102619954,v1:127.0.0.1:6821/4102619954] exists,up 09a997e7-8bfa-4d21-bb95-8853f58aa814 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:21:46.392 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: 1 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: 2 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: 3' 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:21:46.393 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986179 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986179 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986179' 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-he-1 osd.3 26 log_to_monitors true 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435736: 4 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:46.394 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: osd.3 up in weight 1 up_from 30 up_thru 0 down_at 27 last_clean_interval [19,26) [v2:127.0.0.1:6802/4133843623,v1:127.0.0.1:6803/4133843623] [v2:127.0.0.1:6804/4133843623,v1:127.0.0.1:6805/4133843623] exists,up 8e61bc24-46a3-4992-94de-4a83ea303a8a 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:21:46.395 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: 1 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: 2 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: 3' 2026-02-17T06:21:46.396 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:21:46.397 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:46.397 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:21:46.397 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986178 2026-02-17T06:21:46.397 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986178 2026-02-17T06:21:46.397 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986178' 2026-02-17T06:21:46.397 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:47.223 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-handalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:47.223 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: 4 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: 2026-02-17T06:21:45.773+0000 7fdb4c57a8c0 -1 osd.2 26 log_to_monitors true 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: 5 2026-02-17T06:21:47.224 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: osd.2 up in weight 1 up_from 32 up_thru 0 down_at 27 last_clean_interval [15,26) [v2:127.0.0.1:6826/1767850111,v1:127.0.0.1:6827/1767850111] [v2:127.0.0.1:6828/1767850111,v1:127.0.0.1:6829/1767850111] exists,up 60846cb7-a32b-4cd3-8964-55413f0355c6 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:21:47.225 INFO:tasks.workunit.client.0.trial074.stderr:435733: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:21:47.226 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:21:47.227 INFO:tasks.workunit.client.0.trial074.stderr:435733: 1 2026-02-17T06:21:47.227 INFO:tasks.workunit.client.0.trial074.stderr:435733: 2 2026-02-17T06:21:47.227 INFO:tasks.workunit.client.0.trial074.stderr:435733: 3' 2026-02-17T06:21:47.227 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:21:47.227 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:48.820 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:21:48.820 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986180 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986180 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986180' 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672971 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672971 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986180 1-42949672971' 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:21:48.821 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953476 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953476 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986180 1-42949672971 2-137438953476' 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018884 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018884 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986180 1-42949672971 2-137438953476 3-128849018884' 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-133143986180 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:48.822 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-133143986180 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986180 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 133143986180' 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: waiting osd.0 seq 133143986180 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 133143986180 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:48.823 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986180 -lt 133143986180 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/slpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672969 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672969 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986179 1-42949672969' 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953474 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953474 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986179 1-42949672969 2-137438953474' 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T06:21:49.073 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986179 1-42949672969 2-137438953474 3-128849018882' 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-133143986179 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-133143986179 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986179 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 133143986179' 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: waiting osd.0 seq 133143986179 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 133143986179 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 133143986179 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986180 -lt 133143986179 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.074 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672969 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:elpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672970 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672970 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986178 1-42949672970' 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953475 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953475 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986178 1-42949672970 2-137438953475' 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018883 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018883 2026-02-17T06:21:49.077 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-133143986178 1-42949672970 2-137438953475 3-128849018883' 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-133143986178 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-133143986178 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986178 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 133143986178' 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: waiting osd.0 seq 133143986178 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 133143986178 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 133143986178 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986180 -lt 133143986178 2026-02-17T06:21:49.078 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.499 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672970tandalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.499 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672971 2026-02-17T06:21:49.499 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.499 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:21:49.499 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672971 2026-02-17T06:21:49.499 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.499 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672971 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: waiting osd.1 seq 42949672971 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953476 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953476 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.500 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953476 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953476' 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: waiting osd.2 seq 137438953476 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953476 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-128849018884 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-128849018884 2026-02-17T06:21:49.501 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018884 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 128849018884' 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: waiting osd.3 seq 128849018884 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018884 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:49.502 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435733: /ho435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672969 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672969 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672969' 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435730: waiting osd.1 seq 42949672969 2026-02-17T06:21:49.745 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672969 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953474 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953474 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953474 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953474' 2026-02-17T06:21:49.746 INFO:tasks.workunit.client.0.trial074.stderr:435730: waiting osd.2 seq 137438953474 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953474 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-128849018882 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-128849018882 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 128849018882' 2026-02-17T06:21:49.747 INFO:tasks.workunit.client.0.trial074.stderr:435730: waiting osd.3 seq 128849018882 2026-02-17T06:21:49.748 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:21:49.748 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018882 2026-02-17T06:21:49.748 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:21:49.748 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:49.748 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:49.748 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:21:49.748 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:21:49.750 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for 2026-02-17T06:21:49.750 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.750 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:21:49.750 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672970 2026-02-17T06:21:49.750 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.750 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672970 2026-02-17T06:21:49.750 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: waiting osd.1 seq 42949672970 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672970 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953475 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953475 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953475 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953475' 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: waiting osd.2 seq 137438953475 2026-02-17T06:21:49.751 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953475 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-128849018883 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-128849018883 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018883 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 128849018883' 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: waiting osd.3 seq 128849018883 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018883 2026-02-17T06:21:49.752 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:21:49.753 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:49.753 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:49.753 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:21:49.753 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_fome/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:21:49.844 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:21:49.845 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:21:49.845 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:49.845 INFO:tasks.workunit.client.0.trial074.stderr:435733: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:49.845 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:21:49.845 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:21:49.845 INFO:tasks.workunit.client.0.trial074.stderr:435733: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:21:49.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T06:21:50.085 INFO:tasks.workunit.client.0.trial074.stderr:_clean: get_num_active_clean 2026-02-17T06:21:50.085 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:21:50.085 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:21:50.085 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:21:50.086 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:21:50.086 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:21:50.086 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:21:50.086 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:21:50.086 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:50.086 INFO:tasks.workunit.client.0.trial074.stderr:435730: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:50.087 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:21:50.087 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:21:50.087 INFO:tasks.workunit.client.0.trial074.stderr:435730: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:21:50.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T06:21:50.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T06:21:50.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 435728 2026-02-17T06:21:50.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T06:21:50.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 435732 2026-02-17T06:21:50.088 INFO:tasks.workunit.client.0.trial074.stderr:r_clean: get_num_active_clean 2026-02-17T06:21:50.088 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:21:50.088 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:21:50.089 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:21:50.089 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:21:50.089 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:21:50.089 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:21:50.089 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:21:50.090 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:21:50.090 INFO:tasks.workunit.client.0.trial074.stderr:435736: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:21:50.090 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:21:50.090 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:21:50.090 INFO:tasks.workunit.client.0.trial074.stderr:435736: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:21:50.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T06:21:50.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T06:21:50.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T06:21:50.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T06:21:50.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:881: unfound_erasure_coded: return_code=0 2026-02-17T06:21:50.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:882: unfound_erasure_coded: '[' 0 -ne 0 ']' 2026-02-17T06:21:50.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T06:21:50.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T06:21:50.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T06:21:50.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T06:21:50.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:21:50.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: local pg=1.0 2026-02-17T06:21:50.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:888: unfound_erasure_coded: repair 1.0 2026-02-17T06:21:50.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T06:21:50.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T06:21:50.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:21:50.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:21:50.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:21:50.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:21:50.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:50.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T06:21:50.539 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:21:50.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:21:50.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:21:50.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:21:33.736775+0000 '>' 2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:50.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:21:51.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:21:51.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:21:51.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:21:51.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:21:51.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:21:51.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:21:51.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:21:51.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:21:33.736775+0000 '>' 2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:51.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:21:52.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:21:52.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:21:52.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:21:52.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:21:52.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:21:52.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:21:52.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:21:53.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:21:33.736775+0000 '>' 2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:53.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:21:54.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:21:54.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:21:54.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:21:54.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:21:54.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:21:54.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:21:54.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:21:54.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:21:33.736775+0000 '>' 2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:54.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:21:55.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:21:55.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:21:55.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:21:55.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:21:55.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:21:55.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:21:55.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:21:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:21:33.736775+0000 '>' 2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:21:56.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:21:56.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:21:56.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:21:56.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:21:56.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:21:56.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:21:56.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:21:56.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:21:51.180759+0000 '>' 2026-02-17T06:21:33.736775+0000 2026-02-17T06:21:56.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:21:56.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: seq 1 60 2026-02-17T06:21:56.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: for f in `seq 1 60` 2026-02-17T06:21:56.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: ceph -s 2026-02-17T06:21:56.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T06:21:56.645 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:21:56.645 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:21:56.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: break 2026-02-17T06:21:56.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: ceph -s 2026-02-17T06:21:56.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: grep '4 up' 2026-02-17T06:21:56.826 INFO:tasks.workunit.client.0.trial074.stdout: osd: 4 osds: 4 up (since 10s), 4 in (since 32s) 2026-02-17T06:21:56.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: ceph -s 2026-02-17T06:21:56.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: grep '4 in' 2026-02-17T06:21:57.007 INFO:tasks.workunit.client.0.trial074.stdout: osd: 4 osds: 4 up (since 11s), 4 in (since 32s) 2026-02-17T06:21:57.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: ceph -s 2026-02-17T06:21:57.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T06:21:57.197 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:21:57.197 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:21:57.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:21:57.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:21:57.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:21:57.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:21:57.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:21:57.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:21:57.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:21:57.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:21:57.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:21:57.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:21:57.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:21:57.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:21:57.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:21:57.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:21:57.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:21:57.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:21:57.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:21:57.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:21:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:21:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:21:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T06:21:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T06:21:57.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:21:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:21:57.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:21:57.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:21:57.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:21:57.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:21:57.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:21:57.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:21:57.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:21:57.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:21:57.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:21:57.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:21:57.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:21:57.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:21:57.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:21:57.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:21:57.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:21:57.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:21:57.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:21:57.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:21:57.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T06:21:57.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:21:57.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40489 2026-02-17T06:21:57.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:21:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:21:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:21:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T06:21:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_unfound_erasure_coded_overwrites td/osd-scrub-repair 2026-02-17T06:21:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:907: TEST_unfound_erasure_coded_overwrites: '[' true = true ']' 2026-02-17T06:21:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:908: TEST_unfound_erasure_coded_overwrites: unfound_erasure_coded td/osd-scrub-repair true 2026-02-17T06:21:57.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:852: unfound_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:853: unfound_erasure_coded: local allow_overwrites=true 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:854: unfound_erasure_coded: local poolname=ecpool 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:855: unfound_erasure_coded: local payload=ABCDEF 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:857: unfound_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T06:21:57.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T06:21:57.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:21:57.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:57.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:57.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:57.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.583 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:57.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:21:57.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:21:57.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:21:57.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:21:57.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:21:57.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:21:57.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:21:57.613 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:21:57.613 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:21:57.613 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:21:57.613 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:21:57.613 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.613 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.614 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:21:57.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:21:57.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get fsid 2026-02-17T06:21:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:21:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:21:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:21:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:21:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:21:57.671 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:21:57.671 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:21:57.672 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:21:57.672 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:21:57.672 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.672 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.672 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40489/ceph-mon.a.asok 2026-02-17T06:21:57.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:21:57.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40489/ceph-mon.a.asok config get mon_host 2026-02-17T06:21:57.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:858: unfound_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T06:21:57.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T06:21:57.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:21:57.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:21:57.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:21:57.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T06:21:57.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:21:57.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:21:57.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:57.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:57.823 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:57.823 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.823 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:57.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:21:57.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:21:57.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: seq 0 3 2026-02-17T06:21:57.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:21:57.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:21:57.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:57.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:21:57.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:21:57.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:21:57.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:21:57.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:21:57.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:21:57.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f1c0cda6-ef8c-43df-a484-5eed84a216f5 2026-02-17T06:21:57.850 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 f1c0cda6-ef8c-43df-a484-5eed84a216f5 2026-02-17T06:21:57.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 f1c0cda6-ef8c-43df-a484-5eed84a216f5' 2026-02-17T06:21:57.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:21:57.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAFCZRp9TsFMxAAyNOXozkYXYlPB83QSyzkQg== 2026-02-17T06:21:57.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAFCZRp9TsFMxAAyNOXozkYXYlPB83QSyzkQg=="}' 2026-02-17T06:21:57.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f1c0cda6-ef8c-43df-a484-5eed84a216f5 -i td/osd-scrub-repair/0/new.json 2026-02-17T06:21:57.948 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:21:57.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T06:21:57.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAFCZRp9TsFMxAAyNOXozkYXYlPB83QSyzkQg== --osd-uuid f1c0cda6-ef8c-43df-a484-5eed84a216f5 2026-02-17T06:21:57.972 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:57.965+0000 7ff68de478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:57.974 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:57.965+0000 7ff68de478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:57.975 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:57.965+0000 7ff68de478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:21:57.975 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:57.965+0000 7ff68de478c0 -1 bdev(0x560b8af41c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:21:57.975 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:21:57.969+0000 7ff68de478c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T06:22:00.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T06:22:00.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:22:00.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:22:00.578 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:22:00.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:22:00.772 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:22:00.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:22:00.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:22:00.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:22:00.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:22:00.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:22:00.796 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:00.789+0000 7f5a2e85b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:00.798 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:00.789+0000 7f5a2e85b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:00.799 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:00.789+0000 7f5a2e85b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:00.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:22:00.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:22:00.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:22:00.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:22:00.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:22:00.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:00.928 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:22:00.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:22:00.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:00.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:01.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:02.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:02.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:02.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:22:02.085 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:22:02.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:02.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:02.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:02.297 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:02.289+0000 7f5a2e85b8c0 -1 Falling back to public interface 2026-02-17T06:22:03.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:03.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:03.241 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:22:03.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:22:03.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:03.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:03.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:04.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:04.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:04.396 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:22:04.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:22:04.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:04.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:04.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:05.319 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:05.309+0000 7f5a2e85b8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:22:05.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:05.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:05.550 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:22:05.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:22:05.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:05.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:05.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:06.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:06.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:06.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:22:06.703 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:22:06.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:06.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:06.858 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1398545692,v1:127.0.0.1:6803/1398545692] [v2:127.0.0.1:6804/1398545692,v1:127.0.0.1:6805/1398545692] exists,up f1c0cda6-ef8c-43df-a484-5eed84a216f5 2026-02-17T06:22:06.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:22:06.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:22:06.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:22:06.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:22:06.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:22:06.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:06.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:22:06.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:22:06.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:22:06.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T06:22:06.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:22:06.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=241e71b5-f76c-4f43-a1d1-d5310a9b29f0 2026-02-17T06:22:06.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 241e71b5-f76c-4f43-a1d1-d5310a9b29f0' 2026-02-17T06:22:06.867 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 241e71b5-f76c-4f43-a1d1-d5310a9b29f0 2026-02-17T06:22:06.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:22:06.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAOCZRptTUONBAA6uAvmBg33dUhaRzJourkrA== 2026-02-17T06:22:06.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAOCZRptTUONBAA6uAvmBg33dUhaRzJourkrA=="}' 2026-02-17T06:22:06.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 241e71b5-f76c-4f43-a1d1-d5310a9b29f0 -i td/osd-scrub-repair/1/new.json 2026-02-17T06:22:07.029 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:22:07.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T06:22:07.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAOCZRptTUONBAA6uAvmBg33dUhaRzJourkrA== --osd-uuid 241e71b5-f76c-4f43-a1d1-d5310a9b29f0 2026-02-17T06:22:07.056 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:07.049+0000 7f5d138338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:07.058 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:07.049+0000 7f5d138338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:07.059 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:07.049+0000 7f5d138338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:07.059 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:07.053+0000 7f5d138338c0 -1 bdev(0x55f167353c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:22:07.059 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:07.053+0000 7f5d138338c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T06:22:09.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T06:22:09.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:22:09.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:22:09.342 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:22:09.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:22:09.539 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:22:09.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:22:09.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:22:09.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:22:09.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:22:09.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:22:09.563 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:09.557+0000 7f1ce2ca38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:09.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:09.557+0000 7f1ce2ca38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:09.566 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:09.557+0000 7f1ce2ca38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:22:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:22:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:22:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:22:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:22:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:09.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:22:09.693 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:22:09.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:09.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:22:09.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:10.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:10.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:10.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:22:10.848 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:22:10.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:10.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:22:11.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:11.069 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:11.061+0000 7f1ce2ca38c0 -1 Falling back to public interface 2026-02-17T06:22:12.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:12.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:12.004 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:22:12.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:22:12.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:12.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:22:12.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:13.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:13.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:13.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:22:13.157 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:22:13.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:13.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:22:13.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:14.075 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:14.065+0000 7f1ce2ca38c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:22:14.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:14.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:14.307 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:22:14.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:22:14.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:14.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:22:14.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:15.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:15.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:15.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:22:15.464 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:22:15.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:15.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:22:15.616 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/901316154,v1:127.0.0.1:6811/901316154] [v2:127.0.0.1:6812/901316154,v1:127.0.0.1:6813/901316154] exists,up 241e71b5-f76c-4f43-a1d1-d5310a9b29f0 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:22:15.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:22:15.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:22:15.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:22:15.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:22:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a3455bf3-b29a-4c29-9e8c-90002cb257cc 2026-02-17T06:22:15.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a3455bf3-b29a-4c29-9e8c-90002cb257cc' 2026-02-17T06:22:15.624 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 a3455bf3-b29a-4c29-9e8c-90002cb257cc 2026-02-17T06:22:15.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:22:15.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAXCZRp15+UJRAAOgEbtDPLSGvKMjZAWfbBBg== 2026-02-17T06:22:15.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAXCZRp15+UJRAAOgEbtDPLSGvKMjZAWfbBBg=="}' 2026-02-17T06:22:15.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a3455bf3-b29a-4c29-9e8c-90002cb257cc -i td/osd-scrub-repair/2/new.json 2026-02-17T06:22:15.787 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:22:15.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T06:22:15.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAXCZRp15+UJRAAOgEbtDPLSGvKMjZAWfbBBg== --osd-uuid a3455bf3-b29a-4c29-9e8c-90002cb257cc 2026-02-17T06:22:15.815 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:15.805+0000 7ff685c868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:15.816 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:15.809+0000 7ff685c868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:15.817 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:15.809+0000 7ff685c868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:15.818 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:15.809+0000 7ff685c868c0 -1 bdev(0x55e8e60cbc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:22:15.818 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:15.809+0000 7ff685c868c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T06:22:18.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T06:22:18.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:22:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:22:18.086 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:22:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:22:18.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:22:18.280 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:22:18.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:22:18.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:22:18.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:22:18.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:22:18.303 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:18.293+0000 7f24277a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:18.304 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:18.297+0000 7f24277a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:18.305 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:18.297+0000 7f24277a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:18.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:22:18.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:22:18.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:22:18.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:22:18.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:22:18.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:18.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:22:18.431 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:22:18.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:18.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:18.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:19.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:19.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:19.584 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:22:19.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:22:19.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:19.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:19.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:19.837 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:19.829+0000 7f24277a48c0 -1 Falling back to public interface 2026-02-17T06:22:20.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:20.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:20.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:22:20.741 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:22:20.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:20.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:20.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:21.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:21.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:21.892 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:22:21.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:22:21.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:21.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:22.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:22.819 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:22.809+0000 7f24277a48c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:22:23.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:23.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:23.049 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:22:23.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:22:23.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:23.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:23.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:24.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:24.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:24.210 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:22:24.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:22:24.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:24.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:24.361 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3283388734,v1:127.0.0.1:6819/3283388734] [v2:127.0.0.1:6820/3283388734,v1:127.0.0.1:6821/3283388734] exists,up a3455bf3-b29a-4c29-9e8c-90002cb257cc 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 3 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T06:22:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:24.364 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:24.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:22:24.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:22:24.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:22:24.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:22:24.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:22:24.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T06:22:24.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:22:24.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=74d3cda5-8e1e-4627-bc71-3f354fa9145c 2026-02-17T06:22:24.367 INFO:tasks.workunit.client.0.trial074.stdout:add osd3 74d3cda5-8e1e-4627-bc71-3f354fa9145c 2026-02-17T06:22:24.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 74d3cda5-8e1e-4627-bc71-3f354fa9145c' 2026-02-17T06:22:24.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:22:24.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAgCZRppMRZFhAAiUhyci7SpHFzPUMovySIeA== 2026-02-17T06:22:24.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAgCZRppMRZFhAAiUhyci7SpHFzPUMovySIeA=="}' 2026-02-17T06:22:24.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 74d3cda5-8e1e-4627-bc71-3f354fa9145c -i td/osd-scrub-repair/3/new.json 2026-02-17T06:22:24.536 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:22:24.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T06:22:24.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgCZRppMRZFhAAiUhyci7SpHFzPUMovySIeA== --osd-uuid 74d3cda5-8e1e-4627-bc71-3f354fa9145c 2026-02-17T06:22:24.562 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:24.553+0000 7f85261408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:24.563 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:24.557+0000 7f85261408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:24.564 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:24.557+0000 7f85261408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:24.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:24.557+0000 7f85261408c0 -1 bdev(0x5631bc8e7c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T06:22:24.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:24.557+0000 7f85261408c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T06:22:26.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T06:22:26.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:22:26.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T06:22:26.842 INFO:tasks.workunit.client.0.trial074.stdout:adding osd3 key to auth repository 2026-02-17T06:22:26.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:22:27.035 INFO:tasks.workunit.client.0.trial074.stdout:start osd.3 2026-02-17T06:22:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T06:22:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:22:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:22:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:22:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:22:27.059 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:27.049+0000 7f51100068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:27.061 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:27.053+0000 7f51100068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:27.062 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:27.053+0000 7f51100068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:27.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T06:22:27.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:22:27.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T06:22:27.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:22:27.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:22:27.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:27.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:22:27.183 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:22:27.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:27.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:27.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:28.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:28.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:28.335 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:22:28.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:22:28.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:28.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:28.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:28.581 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:28.573+0000 7f51100068c0 -1 Falling back to public interface 2026-02-17T06:22:29.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:29.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:29.491 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:22:29.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:22:29.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:29.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:30.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:30.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:30.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:22:30.646 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:22:30.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:30.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:30.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:31.570 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:31.561+0000 7f51100068c0 -1 osd.3 0 log_to_monitors true 2026-02-17T06:22:31.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:31.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:31.799 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:22:31.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:22:31.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:31.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:31.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:32.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:32.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:32.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:22:32.954 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:22:32.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:32.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:33.106 INFO:tasks.workunit.client.0.trial074.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/92435986,v1:127.0.0.1:6827/92435986] [v2:127.0.0.1:6828/92435986,v1:127.0.0.1:6829/92435986] exists,up 74d3cda5-8e1e-4627-bc71-3f354fa9145c 2026-02-17T06:22:33.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:22:33.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:22:33.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:22:33.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: unfound_erasure_coded: create_ec_pool ecpool true k=2 m=2 2026-02-17T06:22:33.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T06:22:33.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T06:22:33.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T06:22:33.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T06:22:33.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T06:22:33.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T06:22:33.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T06:22:33.609 INFO:tasks.workunit.client.0.trial074.stderr:pool 'ecpool' created 2026-02-17T06:22:33.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:22:34.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T06:22:34.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T06:22:34.838 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 allow_ec_overwrites to true 2026-02-17T06:22:34.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T06:22:34.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:22:34.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:22:34.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:22:34.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:22:34.851 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:22:34.851 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:22:34.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:22:34.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:22:34.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:22:34.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:22:34.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:22:34.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:22:34.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:22:34.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:22:34.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:22:35.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:22:35.131 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:22:35.131 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:22:35.131 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T06:22:35.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:22:35.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:35.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:22:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T06:22:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T06:22:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T06:22:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:35.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:22:35.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T06:22:35.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T06:22:35.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670' 2026-02-17T06:22:35.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:35.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:22:35.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T06:22:35.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T06:22:35.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-60129542148' 2026-02-17T06:22:35.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:35.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:22:35.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378626 2026-02-17T06:22:35.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378626 2026-02-17T06:22:35.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-60129542148 3-81604378626' 2026-02-17T06:22:35.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:35.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T06:22:35.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:35.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:22:35.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T06:22:35.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:35.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T06:22:35.422 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836487 2026-02-17T06:22:35.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T06:22:35.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:35.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T06:22:35.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:22:36.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:22:36.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:36.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T06:22:36.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:36.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T06:22:36.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:36.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:22:36.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T06:22:36.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:36.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T06:22:36.731 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705670 2026-02-17T06:22:36.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T06:22:36.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:22:36.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T06:22:36.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:36.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T06:22:36.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:36.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:22:36.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T06:22:36.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:36.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T06:22:36.887 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542148 2026-02-17T06:22:36.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T06:22:36.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:22:37.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T06:22:37.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:37.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-81604378626 2026-02-17T06:22:37.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:37.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:22:37.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-81604378626 2026-02-17T06:22:37.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:37.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378626 2026-02-17T06:22:37.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 81604378626' 2026-02-17T06:22:37.056 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.3 seq 81604378626 2026-02-17T06:22:37.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:22:37.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378626 -lt 81604378626 2026-02-17T06:22:37.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:22:37.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:37.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:22:37.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:22:37.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:22:37.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:22:37.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:37.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:37.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:22:37.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:22:37.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:22:37.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T06:22:37.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:865: unfound_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T06:22:37.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T06:22:37.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T06:22:37.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T06:22:37.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T06:22:37.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T06:22:37.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T06:22:37.955 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:22:37.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T06:22:38.157 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:22:38.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T06:22:38.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T06:22:38.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T06:22:38.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T06:22:38.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T06:22:38.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T06:22:38.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T06:22:38.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:22:38.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: local primary=1 2026-02-17T06:22:38.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T06:22:38.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: sed -e s/1// 2026-02-17T06:22:38.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T06:22:38.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T06:22:38.349 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T06:22:38.349 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T06:22:38.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T06:22:38.496 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:22:38.497 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:22:38.497 INFO:tasks.workunit.client.0.trial074.stderr:3' 2026-02-17T06:22:38.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2 3 2026-02-17T06:22:38.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: osds=('0' '2' '3') 2026-02-17T06:22:38.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: local -a osds 2026-02-17T06:22:38.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:869: unfound_erasure_coded: local not_primary_first=0 2026-02-17T06:22:38.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:870: unfound_erasure_coded: local not_primary_second=2 2026-02-17T06:22:38.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:871: unfound_erasure_coded: local not_primary_third=3 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:876: unfound_erasure_coded: pids= 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:877: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 447195"' 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 447195' 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:878: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 447196"' 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 447196' 2026-02-17T06:22:38.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:879: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 447198"' 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 447198' 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:880: unfound_erasure_coded: wait_background pids 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 447195 447196 447198' 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 447195 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/447199: /' 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:22:38.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/447201: /' 2026-02-17T06:22:38.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:22:38.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/447203: /' 2026-02-17T06:22:41.002 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.002 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:22:41.002 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T06:22:41.002 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:22:41.003 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: remove 1#1:eb822e21:::SOMETHING:head# 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.004 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:22:41.005 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:22:41.005 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:22:41.005 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T06:22:41.005 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:22:41.005 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:22:41.005 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: 447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T06:22:41.006 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T06:22:41.007 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: remove 3#1:eb822e21:::SOMETHING:head# 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:22:41.008 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:22:41.009 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:22:41.009 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:22:41.009 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T06:22:41.009 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T06:22:41.009 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:22:41.009 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:41.010 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:22:41.011 INFO:tasks.workunit.client.0.trial074.stderr:447199: start osd.0 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:22:41.012 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:22:41.013 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:22:41.014 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:22:41.014 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:22:41.014 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:22:41.014 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T06:22:41.014 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T06:22:41.014 INFO:tasks.workunit.client.0.trial074.stderr:447203: start osd.3 2026-02-17T06:22:41.014 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:22:41.048 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T06:22:41.049 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:22:41.050 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: remove 2#1:eb822e21:::SOMETHING:head# 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T06:22:41.051 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:22:41.052 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:22:41.052 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:41.053 INFO:tasks.workunit.client.0.trial074.stderr:447201: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:22:41.054 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:22:41.055 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T06:22:41.055 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:22:41.055 INFO:tasks.workunit.client.0.trial074.stderr:447201: start osd.2 2026-02-17T06:22:41.055 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=04d560b5-d007-4ffc-b1d0-cb2a7cc97d7e --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40489/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: 2026-02-17T06:22:41.057+0000 7f48c09a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: 2026-02-17T06:22:41.061+0000 7f48c09a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.335 INFO:tasks.workunit.client.0.trial074.stderr:447201: 2026-02-17T06:22:41.061+0000 7f48c09a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: 0 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: 2026-02-17T06:22:42.329+0000 7f48c09a08c0 -1 Falling back to public interface 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:22:45.336 INFO:tasks.workunit.client.0.trial074.stderr:447201: 1 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: 2 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: 3 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:45.337 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.615 INFO:tasks.workunit.client.0.trial074.stderr:447201: 2026-02-17T06:22:45.325+0000 7f48c09a08c0 447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: 2026-02-17T06:22:41.013+0000 7f6df1f9c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: 2026-02-17T06:22:41.013+0000 7f6df1f9c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: 2026-02-17T06:22:41.017+0000 7f6df1f9c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.616 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: 0 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: 1 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: 2026-02-17T06:22:42.537+0000 7f6df1f9c8c0 -1 Falling back to public interface 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.617 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: 2 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: 3 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:45.618 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.663 INFO:tasks.workunit.client.0.trial074.stderr:447203: 2026-02-17T06:22:45.609+0000 7f6df1f9c8c0 447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T06:22:45.663 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:22:45.663 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:22:45.663 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:22:45.663 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: 2026-02-17T06:22:41.013+0000 7fef4aada8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: 2026-02-17T06:22:41.013+0000 7fef4aada8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: 2026-02-17T06:22:41.017+0000 7fef4aada8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:22:45.664 INFO:tasks.workunit.client.0.trial074.stderr:447199: 0 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: 1 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:45.665 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: 2026-02-17T06:22:42.557+0000 7fef4aada8c0 -1 Falling back to public interface 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: 2 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:45.666 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:22:45.667 INFO:tasks.workunit.client.0.trial074.stderr:447199: 3 2026-02-17T06:22:45.667 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:45.667 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:45.667 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:22:46.288 INFO:tasks.workunit.client.0.trial074.stderr:447199: 2026-02-17T06:22:45.653+0000 7fef4aada8c0 -1 osd.3 28 log_to_monitors true 2026-02-17T06:22:46.288 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:46.289 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:46.289 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:22:46.289 INFO:tasks.workunit.client.0.trial074.stderr:447203: 4 2026-02-17T06:22:46.289 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:46.289 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T06:22:46.289 INFO:tasks.workunit.client.0.trial074.stderr:447203: osd.3 up in weight 1 up_from 35 up_thru 0 down_at 29 last_clean_interval [19,28) [v2:127.0.0.1:6818/2911097602,v1:127.0.0.1:6819/2911097602] [v2:127.0.0.1:6820/2911097602,v1:127.0.0.1:6821/2911097602] exists,up 74d3cda5-8e1e-4627-bc71-3f354fa9145c 2026-02-17T06:22:46.289 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:22:46.290 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:22:46.290 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:22:46.290 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:22:46.290 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:22:46.290 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:22:46.290 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:22:46.290 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:22:46.291 INFO:tasks.workunit.client.0.trial074.stderr:447203: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:22:46.291 INFO:tasks.workunit.client.0.trial074.stderr:447203: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:22:46.291 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:22:46.291 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:22:46.291 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:22:46.291 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:22:46.291 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: 1 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: 2 2026-02-17T06:22:46.292 INFO:tasks.workunit.client.0.trial074.stderr:447203: 3' 2026-02-17T06:22:46.293 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:22:46.293 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:46.293 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:22:46.293 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855362 2026-02-17T06:22:46.293 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855362 2026-02-17T06:22:46.293 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362' 2026-02-17T06:22:46.293 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:46.294 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-h-1 osd.0 28 log_to_monitors true 2026-02-17T06:22:46.294 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:46.294 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:46.294 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:22:46.294 INFO:tasks.workunit.client.0.trial074.stderr:447199: 4 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: osd.0 up in weight 1 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6826/1950228221,v1:127.0.0.1:6827/1950228221] [v2:127.0.0.1:6828/1950228221,v1:127.0.0.1:6829/1950228221] exists,up f1c0cda6-ef8c-43df-a484-5eed84a216f5 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:22:46.295 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:22:46.296 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: 1 2026-02-17T06:22:46.297 INFO:tasks.workunit.client.0.trial074.stderr:447199: 2 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: 3' 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855363 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855363 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855363' 2026-02-17T06:22:46.298 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-he-1 osd.2 28 log_to_monitors true 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: 4 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: osd.2 up in weight 1 up_from 32 up_thru 0 down_at 29 last_clean_interval [14,28) [v2:127.0.0.1:6802/1636191573,v1:127.0.0.1:6803/1636191573] [v2:127.0.0.1:6804/1636191573,v1:127.0.0.1:6805/1636191573] exists,up a3455bf3-b29a-4c29-9e8c-90002cb257cc 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:22:46.320 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:22:46.321 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: 1 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: 2 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: 3' 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855364 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855364 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364' 2026-02-17T06:22:46.322 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:48.968 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:22:48.968 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705673 2026-02-17T06:22:48.968 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705673 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-38654705673' 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953474 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953474 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-38654705673 2-137438953474' 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855363 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855363 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-38654705673 2-137438953474 3-150323855363' 2026-02-17T06:22:48.969 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855362 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855362 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855362 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855362' 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: waiting osd.0 seq 150323855362 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 150323855362 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:22:48.970 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:48.971 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 150323855362 2026-02-17T06:22:48.971 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:22:48.971 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:22:48.971 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:48.971 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855362 2026-02-17T06:22:48.971 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:48.971 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705673lpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855363 1-38654705674' 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953475 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953475 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855363 1-38654705674 2-137438953475' 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:48.972 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855362 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855362 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855363 1-38654705674 2-137438953475 3-150323855362' 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855363 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855363 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855363 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855363' 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: waiting osd.0 seq 150323855363 2026-02-17T06:22:48.973 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 150323855363 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 150323855363 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855363 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:48.974 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T06:22:49.005 INFO:tasks.workunit.client.0.trial074.stderr:elpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:22:49.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T06:22:49.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T06:22:49.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364 1-38654705675' 2026-02-17T06:22:49.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:49.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953476 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953476 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364 1-38654705675 2-137438953476' 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855364 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855364 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364 1-38654705675 2-137438953476 3-150323855364' 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855364 2026-02-17T06:22:49.006 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855364 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855364 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855364' 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: waiting osd.0 seq 150323855364 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 150323855364 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 150323855364 2026-02-17T06:22:49.007 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:22:49.008 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:22:49.008 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:22:49.008 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-17T06:22:49.008 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.634 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.634 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:22:49.634 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T06:22:49.634 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.634 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T06:22:49.634 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T06:22:49.634 INFO:tasks.workunit.client.0.trial074.stderr:447199: waiting osd.1 seq 38654705674 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705674 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953475 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953475 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953475 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953475' 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: waiting osd.2 seq 137438953475 2026-02-17T06:22:49.635 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953475 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-150323855362 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-150323855362 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855362 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 150323855362' 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: waiting osd.3 seq 150323855362 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:22:49.636 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855362 2026-02-17T06:22:49.637 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:22:49.637 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:49.637 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:49.637 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:22:49.637 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:22:49.640 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for 2026-02-17T06:22:49.640 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705673 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705673 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: waiting osd.1 seq 38654705673 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705673 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953474 2026-02-17T06:22:49.641 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953474 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953474 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953474' 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: waiting osd.2 seq 137438953474 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953474 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-150323855363 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.642 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-150323855363 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855363 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 150323855363' 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: waiting osd.3 seq 150323855363 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855363 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:22:49.643 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_fo 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: waiting osd.1 seq 38654705675 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:22:49.665 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953476 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953476 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953476 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953476' 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: waiting osd.2 seq 137438953476 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:22:49.666 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953476 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-150323855364 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-150323855364 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855364 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 150323855364' 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: waiting osd.3 seq 150323855364 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-17T06:22:49.667 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:22:49.668 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:49.668 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:49.668 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:22:49.668 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_fo_clean: get_num_active_clean 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:447199: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T06:22:49.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 447196 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:r_clean: get_num_active_clean 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:22:49.980 INFO:tasks.workunit.client.0.trial074.stderr:447203: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:22:49.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:r_clean: get_num_active_clean 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:447201: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 447198 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T06:22:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T06:22:50.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:881: unfound_erasure_coded: return_code=0 2026-02-17T06:22:50.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:882: unfound_erasure_coded: '[' 0 -ne 0 ']' 2026-02-17T06:22:50.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T06:22:50.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T06:22:50.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T06:22:50.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T06:22:50.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:22:50.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: local pg=1.0 2026-02-17T06:22:50.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:888: unfound_erasure_coded: repair 1.0 2026-02-17T06:22:50.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T06:22:50.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T06:22:50.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:22:50.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:22:50.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:22:50.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:22:50.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:50.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T06:22:50.454 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-17T06:22:50.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:50.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:22:50.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:50.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:22:50.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:22:50.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:22:50.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:22:50.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:22:50.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:22:50.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:22:50.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:22:50.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:22:33.606192+0000 '>' 2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:50.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:22:51.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:22:51.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:22:51.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:22:51.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:22:51.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:22:51.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:22:51.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:22:51.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:22:33.606192+0000 '>' 2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:51.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:22:52.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:22:52.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:22:52.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:22:52.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:22:52.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:22:52.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:22:52.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:22:52.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:22:33.606192+0000 '>' 2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:52.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:22:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:22:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:22:53.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:22:53.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:22:53.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:22:53.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:22:53.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:22:54.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:22:33.606192+0000 '>' 2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:54.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:22:55.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:22:55.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:22:55.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:22:55.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:22:55.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:22:55.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:22:55.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:22:55.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:22:33.606192+0000 '>' 2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:55.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:22:56.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:22:56.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:22:56.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:22:56.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:22:56.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:22:56.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:22:56.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:22:56.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:22:51.066127+0000 '>' 2026-02-17T06:22:33.606192+0000 2026-02-17T06:22:56.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:22:56.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: seq 1 60 2026-02-17T06:22:56.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: for f in `seq 1 60` 2026-02-17T06:22:56.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: ceph -s 2026-02-17T06:22:56.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T06:22:56.580 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:22:56.580 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:22:56.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: break 2026-02-17T06:22:56.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: ceph -s 2026-02-17T06:22:56.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: grep '4 up' 2026-02-17T06:22:56.757 INFO:tasks.workunit.client.0.trial074.stdout: osd: 4 osds: 4 up (since 10s), 4 in (since 32s) 2026-02-17T06:22:56.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: ceph -s 2026-02-17T06:22:56.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: grep '4 in' 2026-02-17T06:22:56.936 INFO:tasks.workunit.client.0.trial074.stdout: osd: 4 osds: 4 up (since 11s), 4 in (since 32s) 2026-02-17T06:22:56.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: ceph -s 2026-02-17T06:22:56.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T06:22:57.115 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:22:57.115 INFO:tasks.workunit.client.0.trial074.stdout: 1/1 objects unfound (100.000%) 2026-02-17T06:22:57.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T06:22:57.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:22:57.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:22:57.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:22:57.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:22:57.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:22:57.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:22:57.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:22:57.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:22:57.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:22:57.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:22:57.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:22:57.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:22:57.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:22:57.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:22:57.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:22:57.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:22:57.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:22:57.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:22:57.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:22:57.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:22:57.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:22:57.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:22:57.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:22:57.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:57.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:22:57.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-repair 0 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:22:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:22:57.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:22:57.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:22:57.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:22:57.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:22:57.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:22:57.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:22:57.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:22:57.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:22:57.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:22:57.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:22:57.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:22:57.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:22:57.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:22:57.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T06:22:57.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T06:22:57.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:22:57.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40489 2026-02-17T06:22:57.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40489 2026-02-17T06:22:57.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:22:57.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:22:57.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T06:22:57.270 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T06:22:57.271 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T06:22:57.279 INFO:tasks.workunit:Running workunit scrub/osd-scrub-snaps.sh... 2026-02-17T06:22:57.280 DEBUG:teuthology.orchestra.run.trial074:workunit test scrub/osd-scrub-snaps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh 2026-02-17T06:22:57.336 INFO:tasks.workunit.client.0.trial074.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T06:22:57.341 INFO:tasks.workunit.client.0.trial074.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-snaps 2026-02-17T06:22:57.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:27: run: local dir=td/osd-scrub-snaps 2026-02-17T06:22:57.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:28: run: shift 2026-02-17T06:22:57.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:30: run: export CEPH_MON=127.0.0.1:7121 2026-02-17T06:22:57.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:30: run: CEPH_MON=127.0.0.1:7121 2026-02-17T06:22:57.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:31: run: export CEPH_ARGS 2026-02-17T06:22:57.343 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:32: run: uuidgen 2026-02-17T06:22:57.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:32: run: CEPH_ARGS+='--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none ' 2026-02-17T06:22:57.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:33: run: CEPH_ARGS+='--mon-host=127.0.0.1:7121 ' 2026-02-17T06:22:57.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:35: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T06:22:57.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: set 2026-02-17T06:22:57.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T06:22:57.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: local 'funcs=TEST_scrub_snaps 2026-02-17T06:22:57.346 INFO:tasks.workunit.client.0.trial074.stderr:TEST_scrub_snaps_primary 2026-02-17T06:22:57.346 INFO:tasks.workunit.client.0.trial074.stderr:TEST_scrub_snaps_replica' 2026-02-17T06:22:57.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-17T06:22:57.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-17T06:22:57.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-snaps 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-snaps 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:22:57.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:22:57.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:22:57.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:22:57.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:22:57.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:22:57.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:22:57.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:22:57.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:22:57.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:22:57.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:22:57.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:22:57.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:22:57.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:22:57.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:22:57.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T06:22:57.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:22:57.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:22:57.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.451782 2026-02-17T06:22:57.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:22:57.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:22:57.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-snaps 2026-02-17T06:22:57.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:22:57.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:22:57.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.451782 2026-02-17T06:22:57.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:22:57.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T06:22:57.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T06:22:57.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:22:57.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-17T06:22:57.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps td/osd-scrub-snaps 2026-02-17T06:22:57.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:171: TEST_scrub_snaps: local dir=td/osd-scrub-snaps 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:172: TEST_scrub_snaps: local poolname=test 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:173: TEST_scrub_snaps: local OBJS=16 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:174: TEST_scrub_snaps: local OSDS=1 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:176: TEST_scrub_snaps: TESTDATA=testdata.451782 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:178: TEST_scrub_snaps: run_mon td/osd-scrub-snaps a --osd_pool_default_size=1 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-snaps 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-snaps/a 2026-02-17T06:22:57.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=1 2026-02-17T06:22:57.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:22:57.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:57.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:57.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:57.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:22:57.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:22:57.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 2026-02-17T06:22:57.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:22:57.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:22:57.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:22:57.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:22:57.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:22:57.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:22:57.416 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:22:57.416 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:22:57.416 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:22:57.417 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:22:57.417 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.417 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:22:57.417 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.451782/ceph-mon.a.asok 2026-02-17T06:22:57.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:22:57.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.451782/ceph-mon.a.asok config get fsid 2026-02-17T06:22:57.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:22:57.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:22:57.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:22:57.474 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.451782/ceph-mon.a.asok 2026-02-17T06:22:57.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:22:57.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.451782/ceph-mon.a.asok config get mon_host 2026-02-17T06:22:57.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:179: TEST_scrub_snaps: run_mgr td/osd-scrub-snaps x 2026-02-17T06:22:57.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-snaps 2026-02-17T06:22:57.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:22:57.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:22:57.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:22:57.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-17T06:22:57.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:22:57.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:22:57.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:57.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:57.616 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:57.617 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.617 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:22:57.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:22:57.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:22:57.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:22:57.636 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: expr 1 - 1 2026-02-17T06:22:57.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: seq 0 0 2026-02-17T06:22:57.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:22:57.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:182: TEST_scrub_snaps: run_osd td/osd-scrub-snaps 0 2026-02-17T06:22:57.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:22:57.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:22:57.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:22:57.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T06:22:57.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:22:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=48cc124e-3406-4c74-bb93-0cbae6b7691c 2026-02-17T06:22:57.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 48cc124e-3406-4c74-bb93-0cbae6b7691c' 2026-02-17T06:22:57.645 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 48cc124e-3406-4c74-bb93-0cbae6b7691c 2026-02-17T06:22:57.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:22:57.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBBCZRp54agJhAAZDT5kmbl/oz7/v2zowxP/w== 2026-02-17T06:22:57.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBBCZRp54agJhAAZDT5kmbl/oz7/v2zowxP/w=="}' 2026-02-17T06:22:57.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 48cc124e-3406-4c74-bb93-0cbae6b7691c -i td/osd-scrub-snaps/0/new.json 2026-02-17T06:22:57.738 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:22:57.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-17T06:22:57.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBCZRp54agJhAAZDT5kmbl/oz7/v2zowxP/w== --osd-uuid 48cc124e-3406-4c74-bb93-0cbae6b7691c 2026-02-17T06:22:57.763 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:57.756+0000 7f7b37dc08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:57.764 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:57.760+0000 7f7b37dc08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:57.765 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:57.760+0000 7f7b37dc08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:22:57.765 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:57.760+0000 7f7b37dc08c0 -1 bdev(0x55c67d1f9c00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:22:57.765 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:22:57.760+0000 7f7b37dc08c0 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-17T06:23:00.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-17T06:23:00.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:23:00.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:23:00.369 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:23:00.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:23:00.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:23:00.564 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:23:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:23:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:23:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:23:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:23:00.584 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:00.580+0000 7f48a89488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:23:00.586 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:00.580+0000 7f48a89488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:23:00.587 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:00.580+0000 7f48a89488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:23:00.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:23:00.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:23:00.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:23:00.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:23:00.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:23:00.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:00.714 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:23:00.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:23:00.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:00.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:00.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:23:01.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:23:01.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:01.869 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:23:01.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:23:01.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:01.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:02.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:23:02.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:02.092+0000 7f48a89488c0 -1 Falling back to public interface 2026-02-17T06:23:03.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:23:03.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:03.024 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:23:03.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:23:03.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:03.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:03.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:23:04.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:23:04.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:04.175 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:23:04.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:23:04.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:04.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:04.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:23:05.096 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:05.092+0000 7f48a89488c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:23:05.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:23:05.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:05.327 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:23:05.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:23:05.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:05.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:05.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:23:06.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:23:06.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:06.476 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:23:06.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:23:06.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:06.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:06.628 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1320328119,v1:127.0.0.1:6803/1320328119] [v2:127.0.0.1:6804/1320328119,v1:127.0.0.1:6805/1320328119] exists,up 48cc124e-3406-4c74-bb93-0cbae6b7691c 2026-02-17T06:23:06.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:23:06.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:23:06.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:23:06.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:186: TEST_scrub_snaps: ceph osd set noscrub 2026-02-17T06:23:06.824 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:23:06.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:187: TEST_scrub_snaps: ceph osd set nodeep-scrub 2026-02-17T06:23:07.028 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:23:07.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:190: TEST_scrub_snaps: create_pool test 1 1 2026-02-17T06:23:07.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:23:07.231 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:23:07.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:23:08.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:191: TEST_scrub_snaps: wait_for_clean 2026-02-17T06:23:08.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:23:08.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:23:08.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:23:08.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:23:08.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:23:08.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:23:08.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:23:08.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:23:08.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:23:08.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:23:08.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:23:08.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:23:08.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:23:08.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:23:08.339 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:23:08.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:23:08.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:23:08.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:23:08.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:23:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836482 2026-02-17T06:23:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836482 2026-02-17T06:23:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836482' 2026-02-17T06:23:08.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:23:08.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836482 2026-02-17T06:23:08.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:23:08.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:23:08.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836482 2026-02-17T06:23:08.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:23:08.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836482 2026-02-17T06:23:08.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-17T06:23:08.569 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836482 2026-02-17T06:23:08.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:23:08.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T06:23:08.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:23:09.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:23:09.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:23:09.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T06:23:09.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:23:10.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:23:10.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:23:11.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-17T06:23:11.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:23:11.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:23:11.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:23:11.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:23:11.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:23:11.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:23:11.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:23:11.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:23:11.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:23:11.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:23:11.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:23:11.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:23:11.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:23:11.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:23:11.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:23:11.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:23:11.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:23:11.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:23:11.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: ceph osd dump 2026-02-17T06:23:11.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: grep '^pool.*['\'']test['\'']' 2026-02-17T06:23:11.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: awk '{ print $2 }' 2026-02-17T06:23:11.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: poolid=1 2026-02-17T06:23:11.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:194: TEST_scrub_snaps: dd if=/dev/urandom of=testdata.451782 bs=1032 count=1 2026-02-17T06:23:11.688 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:23:11.689 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:23:11.689 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.2158e-05 s, 19.8 MB/s 2026-02-17T06:23:11.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: seq 1 16 2026-02-17T06:23:11.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj1 testdata.451782 2026-02-17T06:23:11.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj2 testdata.451782 2026-02-17T06:23:11.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj3 testdata.451782 2026-02-17T06:23:11.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj4 testdata.451782 2026-02-17T06:23:11.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj5 testdata.451782 2026-02-17T06:23:11.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj6 testdata.451782 2026-02-17T06:23:11.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj7 testdata.451782 2026-02-17T06:23:11.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj8 testdata.451782 2026-02-17T06:23:11.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj9 testdata.451782 2026-02-17T06:23:11.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj10 testdata.451782 2026-02-17T06:23:11.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj11 testdata.451782 2026-02-17T06:23:11.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj12 testdata.451782 2026-02-17T06:23:11.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj13 testdata.451782 2026-02-17T06:23:11.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:11.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj14 testdata.451782 2026-02-17T06:23:12.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:12.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj15 testdata.451782 2026-02-17T06:23:12.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T06:23:12.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj16 testdata.451782 2026-02-17T06:23:12.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:200: TEST_scrub_snaps: get_primary test obj1 2026-02-17T06:23:12.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:23:12.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:23:12.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:23:12.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:23:12.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:200: TEST_scrub_snaps: local primary=0 2026-02-17T06:23:12.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:202: TEST_scrub_snaps: create_scenario td/osd-scrub-snaps test testdata.451782 0 2026-02-17T06:23:12.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-17T06:23:12.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-17T06:23:12.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.451782 2026-02-17T06:23:12.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=0 2026-02-17T06:23:12.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-17T06:23:12.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-17T06:23:12.281 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap1 2026-02-17T06:23:12.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=1 2026-02-17T06:23:12.283 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:23:12.283 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:23:12.283 INFO:tasks.workunit.client.0.trial074.stderr:256 bytes copied, 4.9412e-05 s, 5.2 MB/s 2026-02-17T06:23:12.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.451782 2026-02-17T06:23:12.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:23:12.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.451782 2026-02-17T06:23:12.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-17T06:23:12.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.451782 2026-02-17T06:23:12.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.451782 2026-02-17T06:23:12.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.451782 2026-02-17T06:23:12.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.451782 2026-02-17T06:23:12.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.451782 2026-02-17T06:23:12.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.451782 2026-02-17T06:23:12.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.451782 2026-02-17T06:23:12.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.451782 2026-02-17T06:23:12.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:23:12.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.451782 2026-02-17T06:23:12.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-17T06:23:12.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-17T06:23:12.582 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap2 2026-02-17T06:23:12.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=2 2026-02-17T06:23:12.585 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records in 2026-02-17T06:23:12.586 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records out 2026-02-17T06:23:12.586 INFO:tasks.workunit.client.0.trial074.stderr:512 bytes copied, 9.3715e-05 s, 5.5 MB/s 2026-02-17T06:23:12.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:23:12.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-17T06:23:12.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-17T06:23:12.684 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap3 2026-02-17T06:23:12.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=3 2026-02-17T06:23:12.686 INFO:tasks.workunit.client.0.trial074.stderr:3+0 records in 2026-02-17T06:23:12.686 INFO:tasks.workunit.client.0.trial074.stderr:3+0 records out 2026-02-17T06:23:12.686 INFO:tasks.workunit.client.0.trial074.stderr:768 bytes copied, 0.000168896 s, 4.5 MB/s 2026-02-17T06:23:12.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.451782 2026-02-17T06:23:12.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-17T06:23:12.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-17T06:23:12.786 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap4 2026-02-17T06:23:12.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=4 2026-02-17T06:23:12.791 INFO:tasks.workunit.client.0.trial074.stderr:4+0 records in 2026-02-17T06:23:12.791 INFO:tasks.workunit.client.0.trial074.stderr:4+0 records out 2026-02-17T06:23:12.791 INFO:tasks.workunit.client.0.trial074.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000248875 s, 4.1 MB/s 2026-02-17T06:23:12.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:23:12.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.451782 2026-02-17T06:23:12.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-17T06:23:12.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-17T06:23:12.888 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap5 2026-02-17T06:23:12.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-17T06:23:12.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-17T06:23:12.990 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap6 2026-02-17T06:23:12.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=6 2026-02-17T06:23:12.994 INFO:tasks.workunit.client.0.trial074.stderr:6+0 records in 2026-02-17T06:23:12.994 INFO:tasks.workunit.client.0.trial074.stderr:6+0 records out 2026-02-17T06:23:12.995 INFO:tasks.workunit.client.0.trial074.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 0.000248054 s, 6.2 MB/s 2026-02-17T06:23:12.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:23:13.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-17T06:23:13.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-17T06:23:13.091 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap7 2026-02-17T06:23:13.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-17T06:23:13.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-17T06:23:13.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-17T06:23:13.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-17T06:23:13.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:23:13.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:23:13.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:23:13.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:23:13.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:23:14.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:23:14.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj1 2026-02-17T06:23:15.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:15.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-17T06:23:15.947 INFO:tasks.workunit.client.0.trial074.stdout:WARNING: only removing head with clones present 2026-02-17T06:23:15.948 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T06:23:16.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T06:23:16.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-17T06:23:17.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:17.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:23:17.966 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:c52c9666:::obj5:2# 2026-02-17T06:23:18.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T06:23:18.502 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-17T06:23:19.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:19.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:19.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:23:20.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:23:20.433 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:20.433 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:23:20.433 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:20.433 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:23:20.434 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:20.435 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:20.436 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:20.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:23:20.437 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:23:20.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:23:21.062 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:c52c9666:::obj5:1# 2026-02-17T06:23:21.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:23:22.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:23:22.684 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:22.684 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:23:22.684 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:22.684 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:23:22.684 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:23:22.684 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:22.685 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:22.686 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:22.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:23:22.688 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:23:22.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T06:23:22.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T06:23:22.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-17T06:23:23.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:23.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=18 2026-02-17T06:23:23.525 INFO:tasks.workunit.client.0.trial074.stderr:18+0 records in 2026-02-17T06:23:23.525 INFO:tasks.workunit.client.0.trial074.stderr:18+0 records out 2026-02-17T06:23:23.525 INFO:tasks.workunit.client.0.trial074.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.000168906 s, 27.3 MB/s 2026-02-17T06:23:23.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:23:24.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj3 2026-02-17T06:23:25.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:25.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=15 2026-02-17T06:23:25.532 INFO:tasks.workunit.client.0.trial074.stderr:15+0 records in 2026-02-17T06:23:25.532 INFO:tasks.workunit.client.0.trial074.stderr:15+0 records out 2026-02-17T06:23:25.533 INFO:tasks.workunit.client.0.trial074.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000105808 s, 36.3 MB/s 2026-02-17T06:23:25.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:23:26.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj4 2026-02-17T06:23:26.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-17T06:23:27.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:27.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:23:28.201 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-17T06:23:28.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:23:29.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:23:29.828 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:29.829 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.830 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.831 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:29.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:23:29.832 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.832 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.832 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.832 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.832 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.832 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.833 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:23:29.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj16 2026-02-17T06:23:29.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-17T06:23:30.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:30.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:23:31.308 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-17T06:23:31.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:23:32.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:23:32.933 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:23:32.933 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:23:32.933 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:23:32.933 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:23:32.934 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:32.935 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:23:32.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:23:32.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T06:23:32.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj2 2026-02-17T06:23:33.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:33.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-17T06:23:34.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:34.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-17T06:23:34.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:34.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=7 2026-02-17T06:23:34.968 INFO:tasks.workunit.client.0.trial074.stderr:7+0 records in 2026-02-17T06:23:34.968 INFO:tasks.workunit.client.0.trial074.stderr:7+0 records out 2026-02-17T06:23:34.968 INFO:tasks.workunit.client.0.trial074.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 0.000242473 s, 7.4 MB/s 2026-02-17T06:23:34.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:23:36.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj6 2026-02-17T06:23:36.486 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:37.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:37.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-17T06:23:38.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj7 2026-02-17T06:23:38.495 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:39.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:39.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-17T06:23:40.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj8 2026-02-17T06:23:40.514 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:41.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:41.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-17T06:23:42.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj9 2026-02-17T06:23:42.543 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:43.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:43.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-17T06:23:44.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj10 2026-02-17T06:23:44.590 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:45.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:45.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-17T06:23:46.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj11 2026-02-17T06:23:46.604 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:47.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:47.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-17T06:23:48.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj12 2026-02-17T06:23:48.618 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:49.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:49.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-17T06:23:50.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj13 2026-02-17T06:23:50.643 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:51.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:51.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-17T06:23:52.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj14 2026-02-17T06:23:52.678 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:53.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:53.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-17T06:23:54.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-17T06:23:54.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj15 2026-02-17T06:23:54.680 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:23:55.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:23:55.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-17T06:23:56.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-17T06:23:56.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-17T06:23:56.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:204: TEST_scrub_snaps: rm -f testdata.451782 2026-02-17T06:23:56.404 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: expr 1 - 1 2026-02-17T06:23:56.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: seq 0 0 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:208: TEST_scrub_snaps: activate_osd td/osd-scrub-snaps 0 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:23:56.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:23:56.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:23:56.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T06:23:56.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T06:23:56.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:23:56.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:23:56.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:23:56.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T06:23:56.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:23:56.409 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:23:56.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:23:56.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-17T06:23:56.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:23:56.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:23:56.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:23:56.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:23:56.430 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:56.424+0000 7f221de838c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:23:56.432 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:56.428+0000 7f221de838c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:23:56.433 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:56.428+0000 7f221de838c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:23:56.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:23:56.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:23:56.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:23:56.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:23:56.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:23:56.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:56.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:23:56.563 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:23:56.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:56.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:56.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:23:57.689 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:23:57.684+0000 7f221de838c0 -1 Falling back to public interface 2026-02-17T06:23:57.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:23:57.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:57.716 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:23:57.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:23:57.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:57.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:57.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:23:58.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:23:58.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:23:58.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:23:58.864 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:23:58.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:23:58.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:23:59.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:00.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:00.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:00.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:24:00.016 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:24:00.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:00.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:00.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:00.687 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:00.684+0000 7f221de838c0 -1 osd.0 17 log_to_monitors true 2026-02-17T06:24:01.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:01.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:01.163 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:24:01.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:24:01.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:01.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:01.309 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 20 up_thru 20 down_at 18 last_clean_interval [5,17) [v2:127.0.0.1:6802/1060522671,v1:127.0.0.1:6803/1060522671] [v2:127.0.0.1:6804/1060522671,v1:127.0.0.1:6805/1060522671] exists,up 48cc124e-3406-4c74-bb93-0cbae6b7691c 2026-02-17T06:24:01.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:24:01.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:24:01.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:24:01.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:210: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 25 2026-02-17T06:24:01.369 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:01.369 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T06:24:01.369 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:01.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:211: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 5 2026-02-17T06:24:01.449 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:01.449 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:24:01.450 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:01.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:212: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-17T06:24:01.525 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:01.525 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T06:24:01.525 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:01.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:213: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-17T06:24:01.601 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:01.601 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T06:24:01.601 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:01.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:216: TEST_scrub_snaps: wait_for_clean 2026-02-17T06:24:01.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:24:01.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:24:01.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:24:01.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:24:01.609 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:24:01.609 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:24:01.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:24:01.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:24:01.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:24:01.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:24:01.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:24:01.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:24:01.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:24:01.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:24:01.720 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:24:01.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:24:01.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:24:01.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:24:01.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:24:01.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345922 2026-02-17T06:24:01.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345922 2026-02-17T06:24:01.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-85899345922' 2026-02-17T06:24:01.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:24:01.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-85899345922 2026-02-17T06:24:01.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:24:01.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:24:01.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-85899345922 2026-02-17T06:24:01.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:24:01.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345922 2026-02-17T06:24:01.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 85899345922' 2026-02-17T06:24:01.948 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 85899345922 2026-02-17T06:24:01.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:24:02.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 85899345922 2026-02-17T06:24:02.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:24:03.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:24:03.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:24:03.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 85899345922 2026-02-17T06:24:03.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:24:04.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:24:04.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:24:04.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345922 -lt 85899345922 2026-02-17T06:24:04.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:24:04.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:24:04.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:24:04.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:24:04.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:24:04.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:24:04.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:24:04.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:24:04.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:24:04.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:24:04.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:24:04.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:24:04.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:24:04.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:24:04.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:24:04.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:24:04.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:24:04.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:24:04.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:218: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_shallow_scrub_chunk_max 2026-02-17T06:24:04.970 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:04.970 INFO:tasks.workunit.client.0.trial074.stdout: "osd_shallow_scrub_chunk_max": "25" 2026-02-17T06:24:04.970 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:04.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:219: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_shallow_scrub_chunk_min 2026-02-17T06:24:05.042 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:05.042 INFO:tasks.workunit.client.0.trial074.stdout: "osd_shallow_scrub_chunk_min": "5" 2026-02-17T06:24:05.043 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:05.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:220: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_pg_stat_report_interval_max_seconds 2026-02-17T06:24:05.111 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:05.111 INFO:tasks.workunit.client.0.trial074.stdout: "osd_pg_stat_report_interval_max_seconds": "1" 2026-02-17T06:24:05.111 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:05.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:221: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_pg_stat_report_interval_max_epochs 2026-02-17T06:24:05.180 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:05.180 INFO:tasks.workunit.client.0.trial074.stdout: "osd_pg_stat_report_interval_max_epochs": "1" 2026-02-17T06:24:05.180 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:05.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:222: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_scrub_chunk_max 2026-02-17T06:24:05.262 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:05.262 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_chunk_max": "15" 2026-02-17T06:24:05.262 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:05.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:223: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_scrub_chunk_min 2026-02-17T06:24:05.337 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:24:05.337 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_chunk_min": "5" 2026-02-17T06:24:05.337 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:24:05.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:225: TEST_scrub_snaps: local pgid=1.0 2026-02-17T06:24:05.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:226: TEST_scrub_snaps: pg_scrub 1.0 2026-02-17T06:24:05.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T06:24:05.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T06:24:05.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T06:24:05.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:24:05.346 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:24:05.346 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:24:05.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:24:05.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:24:05.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:24:05.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:24:05.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:24:05.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:24:05.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:24:05.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:24:05.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:24:05.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T06:24:05.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:24:05.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:24:05.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:24:05.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345924 2026-02-17T06:24:05.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345924 2026-02-17T06:24:05.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-85899345924' 2026-02-17T06:24:05.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:24:05.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-85899345924 2026-02-17T06:24:05.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:24:05.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:24:05.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-85899345924 2026-02-17T06:24:05.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:24:05.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345924 2026-02-17T06:24:05.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 85899345924' 2026-02-17T06:24:05.840 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 85899345924 2026-02-17T06:24:05.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:24:05.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345924 -lt 85899345924 2026-02-17T06:24:05.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:24:05.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T06:24:05.990 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 1.0 loop 0 2026-02-17T06:24:05.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T06:24:05.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T06:24:05.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:24:05.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T06:24:05.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:24:06.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:24:06.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:24:06.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:06.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T06:24:06.349 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:24:06.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:24:06.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:24:06.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:23:07.226230+0000 '>' 2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:06.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:24:07.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:24:07.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:24:07.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:24:07.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:24:07.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:24:07.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:24:07.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:24:07.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:23:07.226230+0000 '>' 2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:07.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:24:08.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:24:08.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:24:08.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:24:08.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:24:08.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:24:08.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:24:08.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:24:08.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:23:07.226230+0000 '>' 2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:08.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:24:09.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:24:09.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:24:09.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:24:09.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:24:09.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:24:09.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:24:09.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:24:09.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:23:07.226230+0000 '>' 2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:09.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:24:10.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:24:10.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:24:10.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:24:10.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:24:10.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:24:10.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:24:10.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:24:11.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:23:07.226230+0000 '>' 2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:11.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:24:12.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:24:12.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:24:12.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:24:12.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:24:12.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:24:12.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:24:12.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:24:12.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:24:06.738255+0000 '>' 2026-02-17T06:23:07.226230+0000 2026-02-17T06:24:12.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:24:12.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:12.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: wc -l 2026-02-17T06:24:12.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: test 2 = 2 2026-02-17T06:24:12.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:232: TEST_scrub_snaps: rados list-inconsistent-pg test 2026-02-17T06:24:12.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:234: TEST_scrub_snaps: jq '. | length' td/osd-scrub-snaps/json 2026-02-17T06:24:12.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:234: TEST_scrub_snaps: test 1 = 1 2026-02-17T06:24:12.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:236: TEST_scrub_snaps: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-17T06:24:12.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:236: TEST_scrub_snaps: test 1.0 = 1.0 2026-02-17T06:24:12.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:238: TEST_scrub_snaps: rados list-inconsistent-obj 1.0 2026-02-17T06:24:12.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:243: TEST_scrub_snaps: jq .inconsistents 2026-02-17T06:24:12.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:243: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:24:12.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:250: TEST_scrub_snaps: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T06:24:12.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:250: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:24:12.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:251: TEST_scrub_snaps: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:24:12.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:24:12.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:253: TEST_scrub_snaps: rados list-inconsistent-snapset 1.0 2026-02-17T06:24:12.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:255: TEST_scrub_snaps: jq .inconsistents 2026-02-17T06:24:12.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:255: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:24:12.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:634: TEST_scrub_snaps: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T06:24:12.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:634: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:24:12.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:635: TEST_scrub_snaps: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:24:12.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:24:12.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:636: TEST_scrub_snaps: test no = yes 2026-02-17T06:24:12.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:641: TEST_scrub_snaps: test '' = yes 2026-02-17T06:24:12.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: find td/osd-scrub-snaps 2026-02-17T06:24:12.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: grep 'osd[^/]*\.pid' 2026-02-17T06:24:12.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: pidfiles=td/osd-scrub-snaps/osd.0.pid 2026-02-17T06:24:12.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:647: TEST_scrub_snaps: pids= 2026-02-17T06:24:12.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:648: TEST_scrub_snaps: for pidfile in ${pidfiles} 2026-02-17T06:24:12.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:650: TEST_scrub_snaps: cat td/osd-scrub-snaps/osd.0.pid 2026-02-17T06:24:12.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:650: TEST_scrub_snaps: pids+='467183 ' 2026-02-17T06:24:12.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:653: TEST_scrub_snaps: ERRORS=0 2026-02-17T06:24:12.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: seq 1 7 2026-02-17T06:24:12.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T06:24:12.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap1 2026-02-17T06:24:12.488 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap1 2026-02-17T06:24:12.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T06:24:12.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap2 2026-02-17T06:24:12.589 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap2 2026-02-17T06:24:12.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T06:24:12.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap3 2026-02-17T06:24:12.690 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap3 2026-02-17T06:24:12.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T06:24:12.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap4 2026-02-17T06:24:12.793 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap4 2026-02-17T06:24:12.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T06:24:12.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap5 2026-02-17T06:24:12.894 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap5 2026-02-17T06:24:12.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T06:24:12.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap6 2026-02-17T06:24:12.997 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap6 2026-02-17T06:24:12.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T06:24:12.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap7 2026-02-17T06:24:13.098 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap7 2026-02-17T06:24:13.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:659: TEST_scrub_snaps: sleep 5 2026-02-17T06:24:18.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:660: TEST_scrub_snaps: local -i loop=0 2026-02-17T06:24:18.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:661: TEST_scrub_snaps: ceph pg dump pgs 2026-02-17T06:24:18.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:661: TEST_scrub_snaps: grep -q snaptrim 2026-02-17T06:24:18.241 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:24:18.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:663: TEST_scrub_snaps: ceph pg dump pgs 2026-02-17T06:24:18.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:663: TEST_scrub_snaps: grep -q snaptrim_error 2026-02-17T06:24:18.401 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:24:18.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:665: TEST_scrub_snaps: break 2026-02-17T06:24:18.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:674: TEST_scrub_snaps: ceph pg dump pgs 2026-02-17T06:24:18.561 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T06:24:18.561 INFO:tasks.workunit.client.0.trial074.stdout:1.0 35 0 0 0 0 23416 0 0 60 0 60 active+clean+inconsistent+snaptrim_error 2026-02-17T06:24:13.095611+0000 23'60 29:140 [0] 0 [0] 0 17'56 2026-02-17T06:24:06.738255+0000 0'0 2026-02-17T06:23:07.226230+0000 7 1 periodic scrub scheduled @ 2026-02-18T17:02:52.042223+0000 33 0 2026-02-17T06:24:18.561 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T06:24:18.561 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T06:24:18.561 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:676: TEST_scrub_snaps: for pid in $pids 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:678: TEST_scrub_snaps: kill -0 467183 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:685: TEST_scrub_snaps: kill_daemons td/osd-scrub-snaps 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:24:18.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:24:26.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:24:26.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:687: TEST_scrub_snaps: declare -a err_strings 2026-02-17T06:24:26.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:688: TEST_scrub_snaps: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj10:.* : is missing in clone_overlap' 2026-02-17T06:24:26.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:689: TEST_scrub_snaps: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : no '\''_'\'' attr' 2026-02-17T06:24:26.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:690: TEST_scrub_snaps: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : is an unexpected clone' 2026-02-17T06:24:26.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:691: TEST_scrub_snaps: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:4 : on disk size [(]4608[)] does not match object info size [(]512[)] adjusted for ondisk to [(]512[)]' 2026-02-17T06:24:26.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:692: TEST_scrub_snaps: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:2' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:693: TEST_scrub_snaps: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:1' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:694: TEST_scrub_snaps: err_strings[6]='log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj5:head : 2 missing clone[(]s[)]' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:695: TEST_scrub_snaps: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj8:head : snaps.seq not set' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:696: TEST_scrub_snaps: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj7:1 : is an unexpected clone' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:697: TEST_scrub_snaps: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj3:head : on disk size [(]3840[)] does not match object info size [(]768[)] adjusted for ondisk to [(]768[)]' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:698: TEST_scrub_snaps: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj6:1 : is an unexpected clone' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:699: TEST_scrub_snaps: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:head : no '\''snapset'\'' attr' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:700: TEST_scrub_snaps: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:7 : clone ignored due to missing snapset' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:701: TEST_scrub_snaps: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:4 : clone ignored due to missing snapset' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:702: TEST_scrub_snaps: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj4:head : expected clone .*:::obj4:7' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:703: TEST_scrub_snaps: err_strings[15]='log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj4:head : 1 missing clone[(]s[)]' 2026-02-17T06:24:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:704: TEST_scrub_snaps: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj1:1 : is an unexpected clone' 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:705: TEST_scrub_snaps: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj9:1 : is missing in clone_size' 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:706: TEST_scrub_snaps: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj11:1 : is an unexpected clone' 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:707: TEST_scrub_snaps: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj14:1 : size 1032 != clone_size 1033' 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:708: TEST_scrub_snaps: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:709: TEST_scrub_snaps: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj15:head : can'\''t decode '\''snapset'\'' attr ' 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:710: TEST_scrub_snaps: err_strings[22]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj10:.* : is missing in clone_overlap' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : no '\''_'\'' attr' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:4 : on disk size [(]4608[)] does not match object info size [(]512[)] adjusted for ondisk to [(]512[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:2' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:1' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj5:head : 2 missing clone[(]s[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj8:head : snaps.seq not set' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj7:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj3:head : on disk size [(]3840[)] does not match object info size [(]768[)] adjusted for ondisk to [(]768[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj6:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:head : no '\''snapset'\'' attr' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:7 : clone ignored due to missing snapset' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:4 : clone ignored due to missing snapset' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj4:head : expected clone .*:::obj4:7' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj4:head : 1 missing clone[(]s[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj1:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj9:1 : is missing in clone_size' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj11:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj14:1 : size 1032 != clone_size 1033' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj15:head : can'\''t decode '\''snapset'\'' attr ' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T06:24:26.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:24:26.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:721: TEST_scrub_snaps: '[' 0 '!=' 0 ']' 2026-02-17T06:24:26.938 INFO:tasks.workunit.client.0.trial074.stdout:TEST PASSED 2026-02-17T06:24:26.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:727: TEST_scrub_snaps: echo 'TEST PASSED' 2026-02-17T06:24:26.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:728: TEST_scrub_snaps: return 0 2026-02-17T06:24:26.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-17T06:24:26.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T06:24:26.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:24:26.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T06:24:26.939 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:24:26.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:24:26.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:24:26.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:24:26.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:24:26.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:24:26.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:24:26.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:24:26.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:24:26.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:24:26.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:24:26.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:24:26.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:24:26.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:24:26.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:24:26.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:24:26.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:24:26.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:24:26.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T06:24:26.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:24:26.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:26.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:26.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.451782 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-snaps 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-snaps 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:24:26.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:24:26.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:24:26.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:24:26.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:24:26.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:24:26.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:24:26.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:24:26.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:24:26.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:24:26.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:24:26.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:24:26.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:24:26.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:24:26.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:24:26.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:24:26.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:24:26.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:24:26.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T06:24:26.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:24:26.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:26.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:26.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.451782 2026-02-17T06:24:26.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:24:26.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:24:26.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-snaps 2026-02-17T06:24:26.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:24:26.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:26.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:26.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.451782 2026-02-17T06:24:26.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:24:26.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:24:26.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:24:26.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-17T06:24:26.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps_primary td/osd-scrub-snaps 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1177: TEST_scrub_snaps_primary: local dir=td/osd-scrub-snaps 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1178: TEST_scrub_snaps_primary: ORIG_ARGS='--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1179: TEST_scrub_snaps_primary: CEPH_ARGS+=' --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1180: TEST_scrub_snaps_primary: _scrub_snaps_multi td/osd-scrub-snaps primary 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:732: _scrub_snaps_multi: local dir=td/osd-scrub-snaps 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:733: _scrub_snaps_multi: local poolname=test 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:734: _scrub_snaps_multi: local OBJS=16 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:735: _scrub_snaps_multi: local OSDS=2 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:736: _scrub_snaps_multi: local which=primary 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:738: _scrub_snaps_multi: TESTDATA=testdata.451782 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:740: _scrub_snaps_multi: run_mon td/osd-scrub-snaps a --osd_pool_default_size=2 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-snaps 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:24:26.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:24:26.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-snaps/a 2026-02-17T06:24:26.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=2 2026-02-17T06:24:27.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:24:27.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:24:27.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:24:27.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:24:27.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:27.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:27.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:24:27.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T06:24:27.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:24:27.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:24:27.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:24:27.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:24:27.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:24:27.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:24:27.037 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:24:27.038 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:24:27.038 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:24:27.039 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:24:27.039 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:27.039 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:27.039 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.451782/ceph-mon.a.asok 2026-02-17T06:24:27.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:24:27.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.451782/ceph-mon.a.asok config get fsid 2026-02-17T06:24:27.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:24:27.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:24:27.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:24:27.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:24:27.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:24:27.096 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:24:27.096 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:24:27.096 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:24:27.097 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:24:27.097 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:27.097 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:27.097 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.451782/ceph-mon.a.asok 2026-02-17T06:24:27.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:24:27.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.451782/ceph-mon.a.asok config get mon_host 2026-02-17T06:24:27.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:741: _scrub_snaps_multi: run_mgr td/osd-scrub-snaps x 2026-02-17T06:24:27.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-snaps 2026-02-17T06:24:27.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:24:27.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:24:27.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:24:27.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-17T06:24:27.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:24:27.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:24:27.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:24:27.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:24:27.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:24:27.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:27.248 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:27.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:24:27.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:24:27.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:24:27.270 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: expr 2 - 1 2026-02-17T06:24:27.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: seq 0 1 2026-02-17T06:24:27.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:24:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 0 2026-02-17T06:24:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:24:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:24:27.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:24:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:24:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T06:24:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:24:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:24:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:24:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:24:27.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:24:27.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T06:24:27.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:24:27.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3256f822-1957-4e8d-ad51-f584645ceea5 2026-02-17T06:24:27.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 3256f822-1957-4e8d-ad51-f584645ceea5' 2026-02-17T06:24:27.281 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 3256f822-1957-4e8d-ad51-f584645ceea5 2026-02-17T06:24:27.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:24:27.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCbCZRpb9gaERAA1tOELmqmSdGfSATRdoaaNQ== 2026-02-17T06:24:27.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCbCZRpb9gaERAA1tOELmqmSdGfSATRdoaaNQ=="}' 2026-02-17T06:24:27.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3256f822-1957-4e8d-ad51-f584645ceea5 -i td/osd-scrub-snaps/0/new.json 2026-02-17T06:24:27.383 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:24:27.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-17T06:24:27.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCbCZRpb9gaERAA1tOELmqmSdGfSATRdoaaNQ== --osd-uuid 3256f822-1957-4e8d-ad51-f584645ceea5 2026-02-17T06:24:27.412 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:27.407+0000 7fcba15248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:27.413 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:27.407+0000 7fcba15248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:27.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:27.407+0000 7fcba15248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:27.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:27.407+0000 7fcba15248c0 -1 bdev(0x555978805c00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:24:27.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:27.407+0000 7fcba15248c0 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-17T06:24:29.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-17T06:24:29.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:24:29.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:24:29.990 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:24:29.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:24:30.189 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:24:30.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:24:30.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:24:30.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:24:30.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:24:30.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:24:30.220 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:30.215+0000 7f3794a678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:30.221 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:30.215+0000 7f3794a678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:30.222 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:30.215+0000 7f3794a678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:30.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:24:30.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:24:30.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:24:30.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:24:30.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:24:30.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:30.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:24:30.351 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:24:30.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:30.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:30.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:24:31.507 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:24:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:31.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:31.721 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:31.715+0000 7f3794a678c0 -1 Falling back to public interface 2026-02-17T06:24:32.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:32.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:24:32.664 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:24:32.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:32.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:32.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:33.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:33.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:33.821 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:24:33.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:24:33.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:33.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:33.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:34.731 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:34.723+0000 7f3794a678c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:24:34.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:34.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:34.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:24:34.976 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:24:34.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:34.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:35.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:36.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:36.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:36.181 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:24:36.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:24:36.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:36.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:24:36.293 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2038552749,v1:127.0.0.1:6803/2038552749] [v2:127.0.0.1:6804/2038552749,v1:127.0.0.1:6805/2038552749] exists,up 3256f822-1957-4e8d-ad51-f584645ceea5 2026-02-17T06:24:36.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:24:36.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:24:36.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 1 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:24:36.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:24:36.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T06:24:36.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:24:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=82258052-0c08-44fb-a7dd-599bafffceab 2026-02-17T06:24:36.298 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 82258052-0c08-44fb-a7dd-599bafffceab 2026-02-17T06:24:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 82258052-0c08-44fb-a7dd-599bafffceab' 2026-02-17T06:24:36.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:24:36.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCkCZRppGNxEhAA82y9l48jrEfPAlJcgR/bZA== 2026-02-17T06:24:36.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCkCZRppGNxEhAA82y9l48jrEfPAlJcgR/bZA=="}' 2026-02-17T06:24:36.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 82258052-0c08-44fb-a7dd-599bafffceab -i td/osd-scrub-snaps/1/new.json 2026-02-17T06:24:36.471 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:24:36.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/1/new.json 2026-02-17T06:24:36.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCkCZRppGNxEhAA82y9l48jrEfPAlJcgR/bZA== --osd-uuid 82258052-0c08-44fb-a7dd-599bafffceab 2026-02-17T06:24:36.502 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:36.495+0000 7f460a2898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:36.503 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:36.495+0000 7f460a2898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:36.504 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:36.499+0000 7f460a2898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:36.505 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:36.499+0000 7f460a2898c0 -1 bdev(0x564e1e723c00 td/osd-scrub-snaps/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:24:36.505 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:36.499+0000 7f460a2898c0 -1 bluestore(td/osd-scrub-snaps/1) _read_fsid unparsable uuid 2026-02-17T06:24:38.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/1/keyring 2026-02-17T06:24:38.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:24:38.785 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:24:38.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:24:38.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:24:38.986 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:24:38.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:24:38.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:24:38.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:24:38.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:24:38.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:24:39.016 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:39.011+0000 7f7de16038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:39.018 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:39.011+0000 7f7de16038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:39.019 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:39.011+0000 7f7de16038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:24:39.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:24:39.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:24:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:24:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:24:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:24:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:24:39.141 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:24:39.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:39.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:24:39.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:40.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:40.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:40.301 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:24:40.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:24:40.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:40.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:24:40.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:40.493 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:40.487+0000 7f7de16038c0 -1 Falling back to public interface 2026-02-17T06:24:41.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:41.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:41.456 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:24:41.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:24:41.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:41.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:24:41.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:42.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:42.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:42.609 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:24:42.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:24:42.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:42.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:24:42.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:43.463 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:24:43.455+0000 7f7de16038c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:24:43.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:43.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:43.768 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:24:43.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:24:43.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:43.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:24:43.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:24:44.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:24:44.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:24:44.919 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:24:44.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:24:44.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:24:44.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:24:45.078 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1246305363,v1:127.0.0.1:6811/1246305363] [v2:127.0.0.1:6812/1246305363,v1:127.0.0.1:6813/1246305363] exists,up 82258052-0c08-44fb-a7dd-599bafffceab 2026-02-17T06:24:45.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:24:45.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:24:45.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:24:45.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:748: _scrub_snaps_multi: ceph osd set noscrub 2026-02-17T06:24:45.276 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:24:45.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:749: _scrub_snaps_multi: ceph osd set nodeep-scrub 2026-02-17T06:24:45.481 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:24:45.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:752: _scrub_snaps_multi: create_pool test 1 1 2026-02-17T06:24:45.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:24:45.683 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:24:45.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:24:46.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:753: _scrub_snaps_multi: wait_for_clean 2026-02-17T06:24:46.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:24:46.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:24:46.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:24:46.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:24:46.697 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:24:46.697 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:24:46.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:24:46.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:24:46.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:24:46.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:24:46.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:24:46.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:24:46.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:24:46.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:24:46.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:24:46.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:24:46.962 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:24:46.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:24:46.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:24:46.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:24:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:24:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:24:47.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:24:47.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:24:47.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:24:47.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T06:24:47.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T06:24:47.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T06:24:47.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:24:47.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:24:47.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:24:47.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:24:47.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:24:47.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:24:47.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:24:47.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:24:47.120 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:24:47.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:24:47.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T06:24:47.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:24:48.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:24:48.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:24:48.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:24:48.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:24:48.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T06:24:48.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:24:48.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:24:48.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T06:24:48.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:24:48.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T06:24:48.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T06:24:48.436 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705666 2026-02-17T06:24:48.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:24:48.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T06:24:48.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:24:48.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:24:48.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:24:48.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:24:48.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:24:48.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:24:48.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:24:48.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:24:49.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:24:49.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:24:49.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:24:49.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: ceph osd dump 2026-02-17T06:24:49.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: grep '^pool.*['\'']test['\'']' 2026-02-17T06:24:49.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: awk '{ print $2 }' 2026-02-17T06:24:49.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: poolid=1 2026-02-17T06:24:49.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:756: _scrub_snaps_multi: dd if=/dev/urandom of=testdata.451782 bs=1032 count=1 2026-02-17T06:24:49.267 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:24:49.268 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:24:49.268 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.4464e-05 s, 29.9 MB/s 2026-02-17T06:24:49.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: seq 1 16 2026-02-17T06:24:49.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj1 testdata.451782 2026-02-17T06:24:49.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj2 testdata.451782 2026-02-17T06:24:49.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj3 testdata.451782 2026-02-17T06:24:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj4 testdata.451782 2026-02-17T06:24:49.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj5 testdata.451782 2026-02-17T06:24:49.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj6 testdata.451782 2026-02-17T06:24:49.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj7 testdata.451782 2026-02-17T06:24:49.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj8 testdata.451782 2026-02-17T06:24:49.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj9 testdata.451782 2026-02-17T06:24:49.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj10 testdata.451782 2026-02-17T06:24:49.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj11 testdata.451782 2026-02-17T06:24:49.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj12 testdata.451782 2026-02-17T06:24:49.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj13 testdata.451782 2026-02-17T06:24:49.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj14 testdata.451782 2026-02-17T06:24:49.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj15 testdata.451782 2026-02-17T06:24:49.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:24:49.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj16 testdata.451782 2026-02-17T06:24:49.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: get_primary test obj1 2026-02-17T06:24:49.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:24:49.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:24:49.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:24:49.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:24:49.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: local primary=1 2026-02-17T06:24:49.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: get_not_primary test obj1 2026-02-17T06:24:49.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=test 2026-02-17T06:24:49.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T06:24:49.859 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary test obj1 2026-02-17T06:24:49.859 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:24:49.859 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:24:49.859 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:24:49.859 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:24:50.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T06:24:50.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map test obj1 2026-02-17T06:24:50.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T06:24:50.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: local replica=0 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: eval create_scenario td/osd-scrub-snaps test testdata.451782 '$primary' 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: create_scenario td/osd-scrub-snaps test testdata.451782 1 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.451782 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=1 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-17T06:24:50.176 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-17T06:24:50.246 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap1 2026-02-17T06:24:50.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=1 2026-02-17T06:24:50.248 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:24:50.248 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:24:50.249 INFO:tasks.workunit.client.0.trial074.stderr:256 bytes copied, 8.024e-05 s, 3.2 MB/s 2026-02-17T06:24:50.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.451782 2026-02-17T06:24:50.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:24:50.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.451782 2026-02-17T06:24:50.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-17T06:24:50.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.451782 2026-02-17T06:24:50.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.451782 2026-02-17T06:24:50.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.451782 2026-02-17T06:24:50.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.451782 2026-02-17T06:24:50.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.451782 2026-02-17T06:24:50.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.451782 2026-02-17T06:24:50.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.451782 2026-02-17T06:24:50.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.451782 2026-02-17T06:24:50.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:24:50.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.451782 2026-02-17T06:24:50.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-17T06:24:50.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-17T06:24:50.567 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap2 2026-02-17T06:24:50.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=2 2026-02-17T06:24:50.570 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records in 2026-02-17T06:24:50.570 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records out 2026-02-17T06:24:50.570 INFO:tasks.workunit.client.0.trial074.stderr:512 bytes copied, 7.9379e-05 s, 6.5 MB/s 2026-02-17T06:24:50.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:24:50.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-17T06:24:50.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-17T06:24:50.669 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap3 2026-02-17T06:24:50.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=3 2026-02-17T06:24:50.672 INFO:tasks.workunit.client.0.trial074.stderr:3+0 records in 2026-02-17T06:24:50.672 INFO:tasks.workunit.client.0.trial074.stderr:3+0 records out 2026-02-17T06:24:50.672 INFO:tasks.workunit.client.0.trial074.stderr:768 bytes copied, 9.3746e-05 s, 8.2 MB/s 2026-02-17T06:24:50.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.451782 2026-02-17T06:24:50.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-17T06:24:50.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-17T06:24:50.771 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap4 2026-02-17T06:24:50.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=4 2026-02-17T06:24:50.775 INFO:tasks.workunit.client.0.trial074.stderr:4+0 records in 2026-02-17T06:24:50.775 INFO:tasks.workunit.client.0.trial074.stderr:4+0 records out 2026-02-17T06:24:50.775 INFO:tasks.workunit.client.0.trial074.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 9.1471e-05 s, 11.2 MB/s 2026-02-17T06:24:50.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:24:50.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.451782 2026-02-17T06:24:50.831 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-17T06:24:50.831 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-17T06:24:50.874 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap5 2026-02-17T06:24:50.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-17T06:24:50.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-17T06:24:50.975 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap6 2026-02-17T06:24:50.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=6 2026-02-17T06:24:50.979 INFO:tasks.workunit.client.0.trial074.stderr:6+0 records in 2026-02-17T06:24:50.979 INFO:tasks.workunit.client.0.trial074.stderr:6+0 records out 2026-02-17T06:24:50.979 INFO:tasks.workunit.client.0.trial074.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 0.000117069 s, 13.1 MB/s 2026-02-17T06:24:50.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:24:51.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-17T06:24:51.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-17T06:24:51.079 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap7 2026-02-17T06:24:51.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-17T06:24:51.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-17T06:24:51.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-17T06:24:51.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-17T06:24:51.159 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:24:51.159 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:24:51.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:24:51.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:24:51.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:24:52.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:24:52.471 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj1 2026-02-17T06:24:53.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:24:53.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-17T06:24:54.000 INFO:tasks.workunit.client.0.trial074.stdout:WARNING: only removing head with clones present 2026-02-17T06:24:54.000 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T06:24:54.533 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-17T06:24:54.533 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-17T06:24:55.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:24:55.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:24:56.036 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:c52c9666:::obj5:2# 2026-02-17T06:24:56.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-17T06:24:56.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-17T06:24:57.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:24:57.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:24:57.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T06:24:58.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:24:58.513 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:24:58.513 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:24:58.513 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:24:58.514 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:24:58.515 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:24:58.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:24:58.516 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:24:58.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:24:59.159 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:c52c9666:::obj5:1# 2026-02-17T06:24:59.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T06:25:00.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:25:00.781 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:25:00.782 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:00.783 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:00.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:25:00.784 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:25:00.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T06:25:00.784 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-17T06:25:00.784 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-17T06:25:01.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:01.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=18 2026-02-17T06:25:01.639 INFO:tasks.workunit.client.0.trial074.stderr:18+0 records in 2026-02-17T06:25:01.639 INFO:tasks.workunit.client.0.trial074.stderr:18+0 records out 2026-02-17T06:25:01.639 INFO:tasks.workunit.client.0.trial074.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.000273401 s, 16.9 MB/s 2026-02-17T06:25:01.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:25:02.810 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj3 2026-02-17T06:25:03.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:03.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=15 2026-02-17T06:25:03.657 INFO:tasks.workunit.client.0.trial074.stderr:15+0 records in 2026-02-17T06:25:03.657 INFO:tasks.workunit.client.0.trial074.stderr:15+0 records out 2026-02-17T06:25:03.657 INFO:tasks.workunit.client.0.trial074.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000101039 s, 38.0 MB/s 2026-02-17T06:25:03.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:25:04.850 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj4 2026-02-17T06:25:04.850 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-17T06:25:05.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:05.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:25:06.328 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-17T06:25:06.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T06:25:07.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:07.953 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.954 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.955 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.955 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.955 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.955 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:25:07.955 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj16 2026-02-17T06:25:07.955 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-17T06:25:08.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:08.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:25:09.432 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-17T06:25:09.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T06:25:11.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:25:11.054 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:25:11.055 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:25:11.056 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:11.056 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:25:11.056 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:11.056 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:25:11.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:25:11.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T06:25:11.057 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj2 2026-02-17T06:25:11.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:11.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-17T06:25:13.081 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:13.081 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-17T06:25:13.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:13.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=7 2026-02-17T06:25:13.085 INFO:tasks.workunit.client.0.trial074.stderr:7+0 records in 2026-02-17T06:25:13.085 INFO:tasks.workunit.client.0.trial074.stderr:7+0 records out 2026-02-17T06:25:13.085 INFO:tasks.workunit.client.0.trial074.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 0.000102661 s, 17.5 MB/s 2026-02-17T06:25:13.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:25:14.257 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj6 2026-02-17T06:25:14.573 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:15.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:15.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-17T06:25:16.293 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj7 2026-02-17T06:25:16.605 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:17.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:17.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-17T06:25:18.326 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj8 2026-02-17T06:25:18.636 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:19.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:19.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-17T06:25:20.342 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj9 2026-02-17T06:25:20.652 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:21.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:21.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-17T06:25:22.374 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj10 2026-02-17T06:25:22.683 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:23.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:23.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-17T06:25:24.434 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj11 2026-02-17T06:25:24.748 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:25.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:25.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-17T06:25:26.469 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj12 2026-02-17T06:25:26.780 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:27.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:27.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-17T06:25:28.486 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj13 2026-02-17T06:25:28.796 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:29.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:29.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-17T06:25:30.498 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj14 2026-02-17T06:25:30.811 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:31.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:31.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-17T06:25:32.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-17T06:25:32.518 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj15 2026-02-17T06:25:32.826 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:25:33.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:25:33.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-17T06:25:34.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-17T06:25:34.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-17T06:25:34.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:767: _scrub_snaps_multi: rm -f testdata.451782 2026-02-17T06:25:34.536 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: expr 2 - 1 2026-02-17T06:25:34.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: seq 0 1 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 0 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T06:25:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:25:34.542 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:25:34.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:25:34.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:25:34.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T06:25:34.544 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:25:34.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:25:34.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:25:34.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-17T06:25:34.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:25:34.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:25:34.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:25:34.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:25:34.571 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:34.563+0000 7f59747428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:25:34.572 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:34.567+0000 7f59747428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:25:34.573 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:34.567+0000 7f59747428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:25:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:25:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:25:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:25:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:25:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:25:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:34.699 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:25:34.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:25:34.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:34.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:25:34.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:35.829 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:35.823+0000 7f59747428c0 -1 Falling back to public interface 2026-02-17T06:25:35.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:35.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:35.854 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:25:35.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:25:35.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:35.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:25:36.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:37.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:37.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:37.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:25:37.015 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:25:37.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:37.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:25:37.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:38.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:38.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:38.172 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:25:38.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:25:38.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:38.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:25:38.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:38.831 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:38.822+0000 7f59747428c0 -1 osd.0 22 log_to_monitors true 2026-02-17T06:25:39.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:39.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:39.330 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:25:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:25:39.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:39.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:25:39.484 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 25 up_thru 25 down_at 23 last_clean_interval [5,22) [v2:127.0.0.1:6802/853558971,v1:127.0.0.1:6803/853558971] [v2:127.0.0.1:6804/853558971,v1:127.0.0.1:6805/853558971] exists,up 3256f822-1957-4e8d-ad51-f584645ceea5 2026-02-17T06:25:39.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:25:39.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:25:39.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 1 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:25:39.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:25:39.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:25:39.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:25:39.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:25:39.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:25:39.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:25:39.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:25:39.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T06:25:39.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:25:39.492 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:25:39.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:25:39.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/1/whoami 2026-02-17T06:25:39.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:25:39.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:25:39.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:25:39.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:25:39.510 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:39.502+0000 7f960f1988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:25:39.511 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:39.502+0000 7f960f1988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:25:39.512 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:39.506+0000 7f960f1988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:25:39.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:25:39.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:25:39.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:25:39.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:25:39.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:25:39.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:39.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:25:39.646 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:25:39.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:39.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:25:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:40.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:40.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:40.802 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:25:40.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:25:40.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:40.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:25:40.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:40.993 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:40.986+0000 7f960f1988c0 -1 Falling back to public interface 2026-02-17T06:25:41.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:41.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:41.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:25:41.957 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:25:41.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:41.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:25:42.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:43.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:43.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:43.112 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:25:43.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:25:43.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:43.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:25:43.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:25:43.991 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:25:43.982+0000 7f960f1988c0 -1 osd.1 22 log_to_monitors true 2026-02-17T06:25:44.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:25:44.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:25:44.267 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:25:44.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:25:44.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:25:44.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:25:44.421 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 23 last_clean_interval [9,22) [v2:127.0.0.1:6810/843829539,v1:127.0.0.1:6811/843829539] [v2:127.0.0.1:6812/843829539,v1:127.0.0.1:6813/843829539] exists,up 82258052-0c08-44fb-a7dd-599bafffceab 2026-02-17T06:25:44.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:25:44.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:25:44.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:25:44.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:774: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-17T06:25:44.491 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:25:44.492 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T06:25:44.492 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.498 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:25:44.498 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T06:25:44.498 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:775: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 3 2026-02-17T06:25:44.584 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:25:44.584 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:25:44.584 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.590 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:25:44.590 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:25:44.590 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:776: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_scrub_chunk_min 3 2026-02-17T06:25:44.676 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:25:44.676 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:25:44.677 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.682 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:25:44.683 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:25:44.683 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:777: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-17T06:25:44.767 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:25:44.767 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T06:25:44.767 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.773 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:25:44.773 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T06:25:44.773 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:778: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-17T06:25:44.862 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:25:44.862 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T06:25:44.862 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.868 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:25:44.868 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T06:25:44.868 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:44.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:779: _scrub_snaps_multi: wait_for_clean 2026-02-17T06:25:44.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:25:44.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:25:44.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:25:44.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:25:44.879 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:25:44.879 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:25:44.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:25:44.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:25:44.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:25:44.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:25:44.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:25:44.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:25:44.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:25:44.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:25:44.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:25:45.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:25:45.131 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:25:45.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:25:45.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:25:45.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:25:45.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182403 2026-02-17T06:25:45.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182403 2026-02-17T06:25:45.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403' 2026-02-17T06:25:45.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:25:45.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:25:45.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T06:25:45.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T06:25:45.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403 1-124554051586' 2026-02-17T06:25:45.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:25:45.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182403 2026-02-17T06:25:45.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:25:45.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:25:45.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182403 2026-02-17T06:25:45.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:25:45.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182403 2026-02-17T06:25:45.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182403' 2026-02-17T06:25:45.286 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 107374182403 2026-02-17T06:25:45.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:25:45.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 107374182403 2026-02-17T06:25:45.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:25:46.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:25:46.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:25:46.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-17T06:25:46.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:25:46.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051586 2026-02-17T06:25:46.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:25:46.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:25:46.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051586 2026-02-17T06:25:46.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:25:46.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T06:25:46.607 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 124554051586 2026-02-17T06:25:46.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-17T06:25:46.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:25:46.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T06:25:46.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:25:46.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:25:46.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:25:46.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:25:46.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:25:46.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:25:46.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:25:46.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:25:46.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:25:46.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:25:46.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:25:47.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:25:47.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:25:47.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:25:47.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:25:47.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:781: _scrub_snaps_multi: local pgid=1.0 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:782: _scrub_snaps_multi: pg_scrub 1.0 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:25:47.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:25:47.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:25:47.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:25:47.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:25:47.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:25:47.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:25:47.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:25:47.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:25:47.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:25:47.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:25:47.726 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:25:47.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:25:47.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:25:47.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:25:47.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182404 2026-02-17T06:25:47.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182404 2026-02-17T06:25:47.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182404' 2026-02-17T06:25:47.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:25:47.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:25:47.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051587 2026-02-17T06:25:47.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051587 2026-02-17T06:25:47.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182404 1-124554051587' 2026-02-17T06:25:47.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:25:47.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182404 2026-02-17T06:25:47.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:25:47.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:25:47.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182404 2026-02-17T06:25:47.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:25:47.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182404 2026-02-17T06:25:47.883 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 107374182404 2026-02-17T06:25:47.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182404' 2026-02-17T06:25:47.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:25:48.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182404 2026-02-17T06:25:48.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:25:49.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:25:49.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:25:49.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182404 2026-02-17T06:25:49.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:25:50.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:25:50.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:25:50.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182405 -lt 107374182404 2026-02-17T06:25:50.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:25:50.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051587 2026-02-17T06:25:50.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:25:50.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:25:50.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051587 2026-02-17T06:25:50.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:25:50.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051587 2026-02-17T06:25:50.361 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 124554051587 2026-02-17T06:25:50.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051587' 2026-02-17T06:25:50.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:25:50.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051588 -lt 124554051587 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 1.0 loop 0 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T06:25:50.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:25:50.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:25:50.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:25:50.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:25:50.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:25:50.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T06:25:50.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:25:50.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:25:50.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:25:50.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:25:50.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:50.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T06:25:50.895 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-17T06:25:50.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:50.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:25:50.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:50.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:25:50.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:25:50.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:25:50.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:25:50.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:25:50.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:25:50.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:25:50.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:25:51.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:24:45.678741+0000 '>' 2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:51.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:25:52.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:25:52.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:25:52.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:25:52.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:25:52.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:25:52.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:25:52.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:25:52.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:24:45.678741+0000 '>' 2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:52.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:25:53.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:25:53.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:25:53.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:25:53.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:25:53.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:25:53.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:25:53.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:25:53.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:24:45.678741+0000 '>' 2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:53.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:25:54.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:25:54.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:25:54.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:25:54.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:25:54.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:25:54.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:25:54.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:25:54.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:24:45.678741+0000 '>' 2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:54.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:25:55.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:25:55.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:25:55.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:25:55.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:25:55.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:25:55.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:25:55.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:25:55.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:25:50.963316+0000 '>' 2026-02-17T06:24:45.678741+0000 2026-02-17T06:25:55.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:25:55.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:25:55.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: wc -l 2026-02-17T06:25:55.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: test 14 -gt 3 2026-02-17T06:25:55.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:25:55.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: wc -l 2026-02-17T06:25:55.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: test 14 -gt 3 2026-02-17T06:25:55.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:789: _scrub_snaps_multi: rados list-inconsistent-pg test 2026-02-17T06:25:55.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: jq '. | length' td/osd-scrub-snaps/json 2026-02-17T06:25:55.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: test 1 = 1 2026-02-17T06:25:55.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-17T06:25:55.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: test 1.0 = 1.0 2026-02-17T06:25:55.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:795: _scrub_snaps_multi: rados list-inconsistent-obj 1.0 --format=json-pretty 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout: "epoch": 29, 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout: "inconsistents": [ 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.752 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "version": 5 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.753 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 1, 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'19", 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'5", 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4189.0:1", 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 5, 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:49.402326+0000", 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:49.402687+0000", 2026-02-17T06:25:55.754 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x86317797", 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.755 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.756 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.757 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 2, 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "version": 20 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.758 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 2, 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'41", 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'20", 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4237.0:1", 2026-02-17T06:25:55.759 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 20, 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.285419+0000", 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.285814+0000", 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.760 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.761 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.762 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 4, 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "version": 42 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch" 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.763 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 4, 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19'45", 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "17'42", 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4273.0:1", 2026-02-17T06:25:55.764 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 42, 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "size": 512, 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.595053+0000", 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.596483+0000", 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x0a8b36d0", 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.765 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.766 INFO:tasks.workunit.client.0.trial074.stdout: "size": 512 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "size": 4608, 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "object_info": { 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:25:55.767 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 4, 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19'45", 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "17'42", 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4273.0:1", 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 42, 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "size": 512, 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.595053+0000", 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.596483+0000", 2026-02-17T06:25:55.768 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x0a8b36d0", 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.769 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj4", 2026-02-17T06:25:55.770 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 7, 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "version": 4 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj4", 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.771 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 7, 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2826278768, 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "version": "22'51", 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'4", 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4186.0:1", 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 4, 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:49.374108+0000", 2026-02-17T06:25:55.772 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:49.374409+0000", 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x86317797", 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.773 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.774 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.775 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 7, 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "version": 0 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "missing", 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "info_missing" 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.776 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: "info_missing" 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1792 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.777 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj1", 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "version": 18 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.778 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj1", 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1828249343, 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'18", 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'1", 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4234.0:1", 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 18, 2026-02-17T06:25:55.779 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.264537+0000", 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.264972+0000", 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.780 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256 2026-02-17T06:25:55.781 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj10", 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.782 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "version": 32 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj10", 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.783 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 718195851, 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'32", 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'10", 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4255.0:1", 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 32, 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.409884+0000", 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.410221+0000", 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.784 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.785 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.786 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.787 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "????", 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.788 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj11", 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "version": 34 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.789 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj11", 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 693400951, 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'34", 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'11", 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4258.0:1", 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 34, 2026-02-17T06:25:55.790 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.430662+0000", 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.431020+0000", 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.791 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.792 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:25:55.793 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj13", 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "version": 38 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.794 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj13", 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2087409765, 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'38", 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'13", 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4264.0:1", 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 38, 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.472427+0000", 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.472853+0000", 2026-02-17T06:25:55.795 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.796 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.797 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]" 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.798 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj14", 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "version": 40 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj14", 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2484217095, 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.799 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'40", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'14", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4267.0:1", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 40, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.493038+0000", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.493428+0000", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.800 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.801 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1033, 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.802 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj15", 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "version": 15 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_corrupted" 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj15", 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 612772309, 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.803 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "version": "15'15", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "0'0", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4219.0:1", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 15, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:49.665206+0000", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:49.665605+0000", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x86317797", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.804 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.805 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_corrupted" 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": "Z2FyYmFnZQo=" 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj2", 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "version": 56 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_missing" 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.806 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj2", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1058988552, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "version": "22'56", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "19'48", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4309.0:1", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 56, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "size": 0, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:51.152816+0000", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:51.153240+0000", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "whiteout", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "dirty" 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xffffffff", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.807 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "size": 0, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 7, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 4, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: 4, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: 3, 2026-02-17T06:25:55.808 INFO:tasks.workunit.client.0.trial074.stdout: 2, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 7, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1024, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: 7, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: 6, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: 5 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_missing" 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: "size": 0 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.809 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj3", 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "version": 44 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch" 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj3", 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1643547569, 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.810 INFO:tasks.workunit.client.0.trial074.stdout: "version": "18'44", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'22", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4279.0:1", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 44, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "size": 768, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.686651+0000", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.687015+0000", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xbe42da58", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.811 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "size": 768 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "size": 3840, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "object_info": { 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj3", 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1643547569, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "version": "18'44", 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'22", 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4279.0:1", 2026-02-17T06:25:55.812 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 44, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "size": 768, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.686651+0000", 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.687015+0000", 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xbe42da58", 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj6", 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.813 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "version": 24 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj6", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2202164420, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'24", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'6", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4243.0:1", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 24, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.329240+0000", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.329571+0000", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.814 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.815 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj7", 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "version": 26 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.816 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj7", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1552453721, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'26", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'7", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4246.0:1", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 26, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.349278+0000", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.349636+0000", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.817 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.818 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj8", 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "version": 28 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj8", 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2381834917, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.819 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'28", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'8", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4249.0:1", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 28, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.369168+0000", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.369482+0000", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.820 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.821 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj9", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "version": 30 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj9", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 3833113727, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'30", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "15'9", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4252.0:1", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 30, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:24:50.389413+0000", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:24:50.389742+0000", 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:25:55.822 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x55c0c7b9", 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.823 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "size": "????", 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:25:55.824 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:25:55.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:797: _scrub_snaps_multi: rados list-inconsistent-snapset 1.0 2026-02-17T06:25:55.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:800: _scrub_snaps_multi: '[' primary = replica ']' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:811: _scrub_snaps_multi: scruberrors=30 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:812: _scrub_snaps_multi: jq .inconsistents 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:812: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1067: _scrub_snaps_multi: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1068: _scrub_snaps_multi: test no = yes 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1073: _scrub_snaps_multi: test '' = yes 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: find td/osd-scrub-snaps 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: grep 'osd[^/]*\.pid' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: pidfiles='td/osd-scrub-snaps/osd.1.pid 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:td/osd-scrub-snaps/osd.0.pid' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1079: _scrub_snaps_multi: pids= 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.1.pid 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='486809 ' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.0.pid 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='486196 ' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1085: _scrub_snaps_multi: ERRORS=0 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1089: _scrub_snaps_multi: '[' primary = primary ']' 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: seq 1 7 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T06:25:55.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap1 2026-02-17T06:25:55.889 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap1 2026-02-17T06:25:55.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T06:25:55.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap2 2026-02-17T06:25:55.990 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap2 2026-02-17T06:25:55.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T06:25:55.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap3 2026-02-17T06:25:56.093 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap3 2026-02-17T06:25:56.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T06:25:56.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap4 2026-02-17T06:25:56.194 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap4 2026-02-17T06:25:56.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T06:25:56.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap5 2026-02-17T06:25:56.297 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap5 2026-02-17T06:25:56.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T06:25:56.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap6 2026-02-17T06:25:56.400 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap6 2026-02-17T06:25:56.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T06:25:56.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap7 2026-02-17T06:25:56.501 INFO:tasks.workunit.client.0.trial074.stdout:removed pool test snap snap7 2026-02-17T06:25:56.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1095: _scrub_snaps_multi: sleep 5 2026-02-17T06:26:01.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1096: _scrub_snaps_multi: local -i loop=0 2026-02-17T06:26:01.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1097: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T06:26:01.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1097: _scrub_snaps_multi: grep -q snaptrim 2026-02-17T06:26:01.646 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:26:01.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1099: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T06:26:01.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1099: _scrub_snaps_multi: grep -q snaptrim_error 2026-02-17T06:26:01.815 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:26:01.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1101: _scrub_snaps_multi: break 2026-02-17T06:26:01.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1111: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T06:26:01.979 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T06:26:01.979 INFO:tasks.workunit.client.0.trial074.stdout:1.0 36 0 0 0 0 24448 0 0 56 0 56 active+clean+inconsistent+snaptrim_error 2026-02-17T06:25:56.498376+0000 22'56 37:133 [1,0] 1 [1,0] 1 22'56 2026-02-17T06:25:50.963316+0000 0'0 2026-02-17T06:24:45.678741+0000 7 1 periodic scrub scheduled @ 2026-02-18T10:08:00.913088+0000 33 0 2026-02-17T06:26:01.979 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T06:26:01.979 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T06:26:01.980 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:26:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T06:26:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 486809 2026-02-17T06:26:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T06:26:01.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 486196 2026-02-17T06:26:01.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1122: _scrub_snaps_multi: kill_daemons td/osd-scrub-snaps 2026-02-17T06:26:01.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:26:01.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:26:01.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:26:01.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:26:01.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:26:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:26:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1124: _scrub_snaps_multi: declare -a err_strings 2026-02-17T06:26:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1125: _scrub_snaps_multi: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' 2026-02-17T06:26:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1126: _scrub_snaps_multi: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' 2026-02-17T06:26:10.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1127: _scrub_snaps_multi: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' 2026-02-17T06:26:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1128: _scrub_snaps_multi: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' 2026-02-17T06:26:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1129: _scrub_snaps_multi: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' 2026-02-17T06:26:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1130: _scrub_snaps_multi: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' 2026-02-17T06:26:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1131: _scrub_snaps_multi: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' 2026-02-17T06:26:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1132: _scrub_snaps_multi: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 30 errors' 2026-02-17T06:26:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:26:10.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 30 errors' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1144: _scrub_snaps_multi: declare -a rep_err_strings 2026-02-17T06:26:10.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: eval echo '$primary' 2026-02-17T06:26:10.326 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: echo 1 2026-02-17T06:26:10.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: osd=1 2026-02-17T06:26:10.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1146: _scrub_snaps_multi: rep_err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-17T06:26:10.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1147: _scrub_snaps_multi: for err_string in "${rep_err_strings[@]}" 2026-02-17T06:26:10.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1149: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:26:10.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1156: _scrub_snaps_multi: '[' 0 '!=' 0 ']' 2026-02-17T06:26:10.329 INFO:tasks.workunit.client.0.trial074.stdout:TEST PASSED 2026-02-17T06:26:10.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1162: _scrub_snaps_multi: echo 'TEST PASSED' 2026-02-17T06:26:10.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1163: _scrub_snaps_multi: return 0 2026-02-17T06:26:10.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1181: TEST_scrub_snaps_primary: err=0 2026-02-17T06:26:10.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1182: TEST_scrub_snaps_primary: CEPH_ARGS='--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T06:26:10.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1183: TEST_scrub_snaps_primary: return 0 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:26:10.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:26:10.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:26:10.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:26:10.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:26:10.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:26:10.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:26:10.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:26:10.339 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:26:10.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:26:10.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:26:10.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:26:10.341 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:26:10.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:26:10.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:26:10.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T06:26:10.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:26:10.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.451782 2026-02-17T06:26:10.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-snaps 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:26:10.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:26:10.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:26:10.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:26:10.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:26:10.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:26:10.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:26:10.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:26:10.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:26:10.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:26:10.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:26:10.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:26:10.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:26:10.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:26:10.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:26:10.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:26:10.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:26:10.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T06:26:10.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:26:10.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.451782 2026-02-17T06:26:10.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:26:10.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:26:10.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-snaps 2026-02-17T06:26:10.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:26:10.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.451782 2026-02-17T06:26:10.372 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps_replica td/osd-scrub-snaps 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1167: TEST_scrub_snaps_replica: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1168: TEST_scrub_snaps_replica: ORIG_ARGS='--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1169: TEST_scrub_snaps_replica: CEPH_ARGS+=' --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1170: TEST_scrub_snaps_replica: _scrub_snaps_multi td/osd-scrub-snaps replica 2026-02-17T06:26:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:732: _scrub_snaps_multi: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:733: _scrub_snaps_multi: local poolname=test 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:734: _scrub_snaps_multi: local OBJS=16 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:735: _scrub_snaps_multi: local OSDS=2 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:736: _scrub_snaps_multi: local which=replica 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:738: _scrub_snaps_multi: TESTDATA=testdata.451782 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:740: _scrub_snaps_multi: run_mon td/osd-scrub-snaps a --osd_pool_default_size=2 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-snaps/a 2026-02-17T06:26:10.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=2 2026-02-17T06:26:10.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:26:10.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:26:10.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:26:10.396 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:26:10.396 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.397 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:26:10.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T06:26:10.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:26:10.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:26:10.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:26:10.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:26:10.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:26:10.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:26:10.427 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:26:10.428 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:26:10.428 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:26:10.428 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:26:10.428 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.428 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.428 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.451782/ceph-mon.a.asok 2026-02-17T06:26:10.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:26:10.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.451782/ceph-mon.a.asok config get fsid 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:26:10.484 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:26:10.485 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:26:10.486 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.486 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.486 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.451782/ceph-mon.a.asok 2026-02-17T06:26:10.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:26:10.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.451782/ceph-mon.a.asok config get mon_host 2026-02-17T06:26:10.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:741: _scrub_snaps_multi: run_mgr td/osd-scrub-snaps x 2026-02-17T06:26:10.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:26:10.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:26:10.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:26:10.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-17T06:26:10.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:26:10.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:26:10.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:26:10.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:26:10.639 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:26:10.639 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.639 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:26:10.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:26:10.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:26:10.661 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: expr 2 - 1 2026-02-17T06:26:10.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: seq 0 1 2026-02-17T06:26:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:26:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 0 2026-02-17T06:26:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:26:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:26:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:26:10.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:26:10.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:26:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:26:10.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:26:10.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:26:10.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:26:10.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:26:10.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:26:10.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T06:26:10.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:26:10.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2b7a125f-3c5b-43e6-9f2f-9251e2be255c 2026-02-17T06:26:10.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2b7a125f-3c5b-43e6-9f2f-9251e2be255c' 2026-02-17T06:26:10.670 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 2b7a125f-3c5b-43e6-9f2f-9251e2be255c 2026-02-17T06:26:10.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:26:10.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQACCpRpaTBXKBAA7XkG1WJp3JVGAYp8aVaXXg== 2026-02-17T06:26:10.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQACCpRpaTBXKBAA7XkG1WJp3JVGAYp8aVaXXg=="}' 2026-02-17T06:26:10.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2b7a125f-3c5b-43e6-9f2f-9251e2be255c -i td/osd-scrub-snaps/0/new.json 2026-02-17T06:26:10.774 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:26:10.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-17T06:26:10.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQACCpRpaTBXKBAA7XkG1WJp3JVGAYp8aVaXXg== --osd-uuid 2b7a125f-3c5b-43e6-9f2f-9251e2be255c 2026-02-17T06:26:10.802 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:10.794+0000 7f01f8a928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:10.804 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:10.798+0000 7f01f8a928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:10.805 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:10.798+0000 7f01f8a928c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:10.806 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:10.798+0000 7f01f8a928c0 -1 bdev(0x55d82fe1fc00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:26:10.806 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:10.798+0000 7f01f8a928c0 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-17T06:26:13.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-17T06:26:13.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:26:13.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:26:13.690 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:26:13.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:26:13.889 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:26:13.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:26:13.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:26:13.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:26:13.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:26:13.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:26:13.920 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:13.914+0000 7f1cd06518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:13.922 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:13.914+0000 7f1cd06518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:13.923 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:13.914+0000 7f1cd06518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:14.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:26:14.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:26:14.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:26:14.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:26:14.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:26:14.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:14.046 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:26:14.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:26:14.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:14.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:26:14.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:15.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:15.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:15.204 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:26:15.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:26:15.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:15.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:26:15.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:15.417 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:15.410+0000 7f1cd06518c0 -1 Falling back to public interface 2026-02-17T06:26:16.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:16.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:16.361 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:26:16.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:26:16.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:16.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:26:16.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:17.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:17.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:17.517 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:26:17.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:26:17.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:17.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:26:17.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:18.387 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:18.378+0000 7f1cd06518c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:26:18.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:18.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:18.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:26:18.676 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:26:18.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:18.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:26:18.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:19.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:19.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:19.834 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:26:19.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:26:19.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:19.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:26:19.987 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/758610159,v1:127.0.0.1:6803/758610159] [v2:127.0.0.1:6804/758610159,v1:127.0.0.1:6805/758610159] exists,up 2b7a125f-3c5b-43e6-9f2f-9251e2be255c 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 1 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:26:19.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:26:19.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:26:19.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:26:19.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:26:19.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:26:19.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:26:19.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:26:19.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T06:26:19.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:26:19.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=fffbfdf7-e86d-48b5-936e-3c3931b264c1 2026-02-17T06:26:19.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 fffbfdf7-e86d-48b5-936e-3c3931b264c1' 2026-02-17T06:26:19.995 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 fffbfdf7-e86d-48b5-936e-3c3931b264c1 2026-02-17T06:26:19.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:26:20.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAMCpRpplc6ABAAjxSFaP+TdTEqIsxJpkwSxQ== 2026-02-17T06:26:20.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAMCpRpplc6ABAAjxSFaP+TdTEqIsxJpkwSxQ=="}' 2026-02-17T06:26:20.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new fffbfdf7-e86d-48b5-936e-3c3931b264c1 -i td/osd-scrub-snaps/1/new.json 2026-02-17T06:26:20.162 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:26:20.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/1/new.json 2026-02-17T06:26:20.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMCpRpplc6ABAAjxSFaP+TdTEqIsxJpkwSxQ== --osd-uuid fffbfdf7-e86d-48b5-936e-3c3931b264c1 2026-02-17T06:26:20.200 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:20.194+0000 7fc177a2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:20.201 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:20.194+0000 7fc177a2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:20.203 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:20.194+0000 7fc177a2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:20.203 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:20.194+0000 7fc177a2d8c0 -1 bdev(0x55c841a25c00 td/osd-scrub-snaps/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:26:20.203 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:20.194+0000 7fc177a2d8c0 -1 bluestore(td/osd-scrub-snaps/1) _read_fsid unparsable uuid 2026-02-17T06:26:22.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/1/keyring 2026-02-17T06:26:22.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:26:22.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:26:22.467 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:26:22.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:26:22.671 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:26:22.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:26:22.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:26:22.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:26:22.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:26:22.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:26:22.690 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:22.682+0000 7f5c894458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:22.691 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:22.682+0000 7f5c894458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:22.693 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:22.686+0000 7f5c894458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:26:22.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:26:22.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:26:22.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:26:22.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:26:22.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:26:22.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:22.827 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:26:22.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:26:22.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:22.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:26:22.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:23.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:23.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:23.983 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:26:23.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:26:23.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:23.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:26:24.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:24.165 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:24.158+0000 7f5c894458c0 -1 Falling back to public interface 2026-02-17T06:26:25.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:25.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:25.140 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:26:25.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:26:25.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:25.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:26:25.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:26.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:26.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:26.296 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:26:26.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:26:26.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:26.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:26:26.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:27.133 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:26:27.126+0000 7f5c894458c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:26:27.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:27.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:27.451 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:26:27.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:26:27.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:27.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:26:27.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:26:28.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:26:28.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:26:28.603 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:26:28.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:26:28.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:26:28.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:26:28.757 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3914026501,v1:127.0.0.1:6811/3914026501] [v2:127.0.0.1:6812/3914026501,v1:127.0.0.1:6813/3914026501] exists,up fffbfdf7-e86d-48b5-936e-3c3931b264c1 2026-02-17T06:26:28.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:26:28.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:26:28.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:26:28.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:748: _scrub_snaps_multi: ceph osd set noscrub 2026-02-17T06:26:29.001 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:26:29.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:749: _scrub_snaps_multi: ceph osd set nodeep-scrub 2026-02-17T06:26:29.203 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:26:29.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:752: _scrub_snaps_multi: create_pool test 1 1 2026-02-17T06:26:29.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:26:29.406 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:26:29.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:26:30.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:753: _scrub_snaps_multi: wait_for_clean 2026-02-17T06:26:30.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:26:30.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:26:30.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:26:30.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:26:30.419 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:26:30.419 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:26:30.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:26:30.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:26:30.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:26:30.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:26:30.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:26:30.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:26:30.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:26:30.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:26:30.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:26:30.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:26:30.666 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:26:30.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:26:30.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:26:30.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:26:30.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T06:26:30.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T06:26:30.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T06:26:30.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:26:30.743 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:26:30.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T06:26:30.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T06:26:30.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T06:26:30.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:26:30.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T06:26:30.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:26:30.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:26:30.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T06:26:30.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:26:30.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T06:26:30.821 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836484 2026-02-17T06:26:30.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T06:26:30.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:26:30.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T06:26:30.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:26:30.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T06:26:30.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:26:30.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:26:30.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T06:26:30.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:26:30.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T06:26:30.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T06:26:30.979 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672962 2026-02-17T06:26:30.979 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:26:31.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T06:26:31.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:26:31.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:26:31.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:26:31.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:26:31.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:26:31.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:26:31.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:26:31.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:26:31.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:26:31.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:26:31.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:26:31.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:26:31.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:26:31.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:26:31.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:26:31.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:26:31.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:26:31.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:26:31.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: ceph osd dump 2026-02-17T06:26:31.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: grep '^pool.*['\'']test['\'']' 2026-02-17T06:26:31.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: awk '{ print $2 }' 2026-02-17T06:26:31.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: poolid=1 2026-02-17T06:26:31.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:756: _scrub_snaps_multi: dd if=/dev/urandom of=testdata.451782 bs=1032 count=1 2026-02-17T06:26:31.841 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:26:31.841 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:26:31.842 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.6458e-05 s, 28.3 MB/s 2026-02-17T06:26:31.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: seq 1 16 2026-02-17T06:26:31.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:31.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj1 testdata.451782 2026-02-17T06:26:31.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:31.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj2 testdata.451782 2026-02-17T06:26:31.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:31.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj3 testdata.451782 2026-02-17T06:26:31.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:31.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj4 testdata.451782 2026-02-17T06:26:31.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:31.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj5 testdata.451782 2026-02-17T06:26:31.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:31.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj6 testdata.451782 2026-02-17T06:26:32.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj7 testdata.451782 2026-02-17T06:26:32.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj8 testdata.451782 2026-02-17T06:26:32.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj9 testdata.451782 2026-02-17T06:26:32.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj10 testdata.451782 2026-02-17T06:26:32.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj11 testdata.451782 2026-02-17T06:26:32.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj12 testdata.451782 2026-02-17T06:26:32.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj13 testdata.451782 2026-02-17T06:26:32.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj14 testdata.451782 2026-02-17T06:26:32.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj15 testdata.451782 2026-02-17T06:26:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T06:26:32.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj16 testdata.451782 2026-02-17T06:26:32.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: get_primary test obj1 2026-02-17T06:26:32.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:26:32.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:26:32.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:26:32.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:26:32.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: local primary=1 2026-02-17T06:26:32.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: get_not_primary test obj1 2026-02-17T06:26:32.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=test 2026-02-17T06:26:32.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T06:26:32.453 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary test obj1 2026-02-17T06:26:32.453 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:26:32.453 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:26:32.453 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:26:32.453 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:26:32.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T06:26:32.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map test obj1 2026-02-17T06:26:32.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T06:26:32.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: local replica=0 2026-02-17T06:26:32.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: eval create_scenario td/osd-scrub-snaps test testdata.451782 '$replica' 2026-02-17T06:26:32.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: create_scenario td/osd-scrub-snaps test testdata.451782 0 2026-02-17T06:26:32.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-17T06:26:32.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-17T06:26:32.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.451782 2026-02-17T06:26:32.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=0 2026-02-17T06:26:32.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-17T06:26:32.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-17T06:26:32.841 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap1 2026-02-17T06:26:32.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=1 2026-02-17T06:26:32.844 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:26:32.844 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:26:32.844 INFO:tasks.workunit.client.0.trial074.stderr:256 bytes copied, 5.1796e-05 s, 4.9 MB/s 2026-02-17T06:26:32.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.451782 2026-02-17T06:26:32.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:26:32.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.451782 2026-02-17T06:26:32.931 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-17T06:26:32.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:32.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.451782 2026-02-17T06:26:32.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:32.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.451782 2026-02-17T06:26:32.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:32.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.451782 2026-02-17T06:26:33.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:33.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.451782 2026-02-17T06:26:33.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:33.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.451782 2026-02-17T06:26:33.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:33.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.451782 2026-02-17T06:26:33.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:33.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.451782 2026-02-17T06:26:33.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:33.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.451782 2026-02-17T06:26:33.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T06:26:33.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.451782 2026-02-17T06:26:33.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-17T06:26:33.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-17T06:26:33.259 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap2 2026-02-17T06:26:33.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=2 2026-02-17T06:26:33.263 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records in 2026-02-17T06:26:33.263 INFO:tasks.workunit.client.0.trial074.stderr:2+0 records out 2026-02-17T06:26:33.263 INFO:tasks.workunit.client.0.trial074.stderr:512 bytes copied, 8.9758e-05 s, 5.7 MB/s 2026-02-17T06:26:33.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:26:33.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-17T06:26:33.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-17T06:26:33.360 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap3 2026-02-17T06:26:33.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=3 2026-02-17T06:26:33.364 INFO:tasks.workunit.client.0.trial074.stderr:3+0 records in 2026-02-17T06:26:33.364 INFO:tasks.workunit.client.0.trial074.stderr:3+0 records out 2026-02-17T06:26:33.364 INFO:tasks.workunit.client.0.trial074.stderr:768 bytes copied, 9.4396e-05 s, 8.1 MB/s 2026-02-17T06:26:33.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.451782 2026-02-17T06:26:33.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-17T06:26:33.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-17T06:26:33.463 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap4 2026-02-17T06:26:33.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=4 2026-02-17T06:26:33.466 INFO:tasks.workunit.client.0.trial074.stderr:4+0 records in 2026-02-17T06:26:33.466 INFO:tasks.workunit.client.0.trial074.stderr:4+0 records out 2026-02-17T06:26:33.466 INFO:tasks.workunit.client.0.trial074.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 9.2763e-05 s, 11.0 MB/s 2026-02-17T06:26:33.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:26:33.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.451782 2026-02-17T06:26:33.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-17T06:26:33.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-17T06:26:33.565 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap5 2026-02-17T06:26:33.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-17T06:26:33.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-17T06:26:33.668 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap6 2026-02-17T06:26:33.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=6 2026-02-17T06:26:33.672 INFO:tasks.workunit.client.0.trial074.stderr:6+0 records in 2026-02-17T06:26:33.672 INFO:tasks.workunit.client.0.trial074.stderr:6+0 records out 2026-02-17T06:26:33.672 INFO:tasks.workunit.client.0.trial074.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 0.000114184 s, 13.5 MB/s 2026-02-17T06:26:33.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.451782 2026-02-17T06:26:33.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-17T06:26:33.701 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-17T06:26:33.770 INFO:tasks.workunit.client.0.trial074.stdout:created pool test snap snap7 2026-02-17T06:26:33.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-17T06:26:33.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-17T06:26:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-17T06:26:33.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-17T06:26:33.850 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:26:33.850 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:26:33.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:26:33.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:26:33.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:26:35.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:26:35.161 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj1 2026-02-17T06:26:36.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:36.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-17T06:26:36.664 INFO:tasks.workunit.client.0.trial074.stdout:WARNING: only removing head with clones present 2026-02-17T06:26:36.665 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T06:26:37.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T06:26:37.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-17T06:26:38.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:38.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:26:38.692 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:c52c9666:::obj5:2# 2026-02-17T06:26:39.238 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T06:26:39.238 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-17T06:26:40.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:40.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:40.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:26:41.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:26:41.166 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:41.166 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:26:41.166 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:41.166 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:26:41.166 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:26:41.166 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:26:41.166 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:41.167 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:41.168 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:41.169 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:41.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:26:41.170 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:26:41.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:26:41.804 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:c52c9666:::obj5:1# 2026-02-17T06:26:42.341 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:26:43.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:26:43.429 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:43.430 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T06:26:43.431 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:43.431 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:43.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:26:43.431 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:26:43.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T06:26:43.433 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T06:26:43.433 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-17T06:26:44.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:44.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=18 2026-02-17T06:26:44.302 INFO:tasks.workunit.client.0.trial074.stderr:18+0 records in 2026-02-17T06:26:44.302 INFO:tasks.workunit.client.0.trial074.stderr:18+0 records out 2026-02-17T06:26:44.302 INFO:tasks.workunit.client.0.trial074.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.00017689 s, 26.1 MB/s 2026-02-17T06:26:44.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:26:45.478 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj3 2026-02-17T06:26:46.328 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:46.328 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=15 2026-02-17T06:26:46.329 INFO:tasks.workunit.client.0.trial074.stderr:15+0 records in 2026-02-17T06:26:46.329 INFO:tasks.workunit.client.0.trial074.stderr:15+0 records out 2026-02-17T06:26:46.329 INFO:tasks.workunit.client.0.trial074.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000217516 s, 17.7 MB/s 2026-02-17T06:26:46.329 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:26:47.511 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj4 2026-02-17T06:26:47.511 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-17T06:26:48.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:48.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:26:49.009 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-17T06:26:49.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:26:50.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:26:50.625 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:26:50.625 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.625 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:26:50.625 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:26:50.625 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:26:50.625 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:26:50.625 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:50.626 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:50.627 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.628 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:50.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:26:50.628 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.628 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.629 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.629 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.629 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.629 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.629 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T06:26:50.629 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj16 2026-02-17T06:26:50.629 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-17T06:26:51.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:51.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T06:26:52.109 INFO:tasks.workunit.client.0.trial074.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-17T06:26:52.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T06:26:53.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T06:26:53.734 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:53.735 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T06:26:53.736 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:53.736 INFO:tasks.workunit.client.0.trial074.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T06:26:53.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T06:26:53.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T06:26:53.736 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj2 2026-02-17T06:26:54.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:54.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-17T06:26:55.770 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:55.771 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-17T06:26:55.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:55.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.451782 bs=256 count=7 2026-02-17T06:26:55.773 INFO:tasks.workunit.client.0.trial074.stderr:7+0 records in 2026-02-17T06:26:55.773 INFO:tasks.workunit.client.0.trial074.stderr:7+0 records out 2026-02-17T06:26:55.773 INFO:tasks.workunit.client.0.trial074.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 7.0742e-05 s, 25.3 MB/s 2026-02-17T06:26:55.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.451782 2026-02-17T06:26:56.982 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj6 2026-02-17T06:26:57.296 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:26:57.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:57.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-17T06:26:58.991 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj7 2026-02-17T06:26:59.308 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:26:59.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:26:59.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-17T06:27:01.018 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj8 2026-02-17T06:27:01.345 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:01.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:01.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-17T06:27:03.042 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj9 2026-02-17T06:27:03.353 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:03.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:03.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-17T06:27:05.086 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj10 2026-02-17T06:27:05.405 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:05.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:05.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-17T06:27:07.106 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj11 2026-02-17T06:27:07.423 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:07.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:07.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-17T06:27:09.122 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj12 2026-02-17T06:27:09.431 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:09.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:09.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-17T06:27:11.158 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj13 2026-02-17T06:27:11.467 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:12.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:12.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-17T06:27:13.162 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj14 2026-02-17T06:27:13.483 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:14.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:14.015 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-17T06:27:15.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-17T06:27:15.194 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj15 2026-02-17T06:27:15.508 INFO:tasks.workunit.client.0.trial074.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T06:27:16.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:27:16.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-17T06:27:17.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-17T06:27:17.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-17T06:27:17.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:767: _scrub_snaps_multi: rm -f testdata.451782 2026-02-17T06:27:17.224 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: expr 2 - 1 2026-02-17T06:27:17.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: seq 0 1 2026-02-17T06:27:17.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:27:17.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 0 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T06:27:17.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:17.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:27:17.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:27:17.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:27:17.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:27:17.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:27:17.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:27:17.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:27:17.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:27:17.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:27:17.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:27:17.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:27:17.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:27:17.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T06:27:17.233 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:27:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T06:27:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:27:17.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-17T06:27:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T06:27:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:27:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:27:17.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:27:17.257 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:17.249+0000 7fc5af0058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:17.258 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:17.249+0000 7fc5af0058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:17.259 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:17.249+0000 7fc5af0058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:17.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T06:27:17.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:27:17.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:27:17.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:27:17.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:27:17.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:17.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:27:17.385 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:27:17.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:17.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:17.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:18.502 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:18.493+0000 7fc5af0058c0 -1 Falling back to public interface 2026-02-17T06:27:18.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:18.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:18.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:27:18.540 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:27:18.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:18.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:18.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:19.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:19.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:19.696 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:27:19.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:27:19.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:19.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:19.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:20.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:20.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:20.853 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:27:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:27:20.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:20.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:21.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:21.487 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:21.477+0000 7fc5af0058c0 -1 osd.0 23 log_to_monitors true 2026-02-17T06:27:22.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:22.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:22.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:27:22.012 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:27:22.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:22.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:22.166 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 26 up_thru 26 down_at 24 last_clean_interval [5,23) [v2:127.0.0.1:6802/552633357,v1:127.0.0.1:6803/552633357] [v2:127.0.0.1:6804/552633357,v1:127.0.0.1:6805/552633357] exists,up 2b7a125f-3c5b-43e6-9f2f-9251e2be255c 2026-02-17T06:27:22.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:27:22.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:27:22.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:27:22.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:27:22.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 1 2026-02-17T06:27:22.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:27:22.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T06:27:22.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T06:27:22.173 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:27:22.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:27:22.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.451782/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:27:22.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/1/whoami 2026-02-17T06:27:22.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:27:22.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:27:22.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:27:22.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:27:22.197 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:22.189+0000 7f7cdce518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:22.198 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:22.189+0000 7f7cdce518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:22.199 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:22.189+0000 7f7cdce518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:22.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:27:22.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:27:22.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:27:22.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:27:22.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:27:22.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:22.326 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:27:22.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:27:22.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:22.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:22.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:23.450 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:23.441+0000 7f7cdce518c0 -1 Falling back to public interface 2026-02-17T06:27:23.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:23.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:23.484 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:27:23.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:27:23.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:23.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:23.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:24.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:24.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:24.638 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:27:24.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:27:24.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:24.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:24.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:25.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:25.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:25.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:27:25.799 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:27:25.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:25.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:25.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:26.423 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:26.413+0000 7f7cdce518c0 -1 osd.1 23 log_to_monitors true 2026-02-17T06:27:26.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:26.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:26.957 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:27:26.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:27:26.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:26.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:27.112 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 24 last_clean_interval [10,23) [v2:127.0.0.1:6810/2387355246,v1:127.0.0.1:6811/2387355246] [v2:127.0.0.1:6812/2387355246,v1:127.0.0.1:6813/2387355246] exists,up fffbfdf7-e86d-48b5-936e-3c3931b264c1 2026-02-17T06:27:27.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:27:27.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:27:27.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:27:27.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:774: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-17T06:27:27.184 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:27:27.184 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T06:27:27.184 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.190 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:27:27.190 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T06:27:27.191 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:775: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 3 2026-02-17T06:27:27.276 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:27:27.277 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:27:27.277 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.283 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:27:27.283 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:27:27.283 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:776: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_scrub_chunk_min 3 2026-02-17T06:27:27.368 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:27:27.369 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:27:27.369 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.375 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:27:27.375 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T06:27:27.375 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:777: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-17T06:27:27.458 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:27:27.458 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T06:27:27.458 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.464 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:27:27.464 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T06:27:27.464 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:778: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-17T06:27:27.552 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:27:27.552 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T06:27:27.552 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.558 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:27:27.558 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T06:27:27.558 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:27.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:779: _scrub_snaps_multi: wait_for_clean 2026-02-17T06:27:27.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:27:27.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:27:27.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:27:27.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:27:27.567 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:27:27.567 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:27:27.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:27:27.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:27:27.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:27:27.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:27:27.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:27:27.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:27:27.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:27:27.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:27:27.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:27:27.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:27:27.838 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:27:27.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:27:27.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:27:27.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:27:27.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149699 2026-02-17T06:27:27.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149699 2026-02-17T06:27:27.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699' 2026-02-17T06:27:27.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:27:27.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:27:27.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T06:27:27.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T06:27:27.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149699 1-124554051586' 2026-02-17T06:27:27.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:27:27.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149699 2026-02-17T06:27:27.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:27:27.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:27:27.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149699 2026-02-17T06:27:27.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:27:27.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149699 2026-02-17T06:27:27.994 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 111669149699 2026-02-17T06:27:27.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149699' 2026-02-17T06:27:27.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:27:28.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149698 -lt 111669149699 2026-02-17T06:27:28.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:27:29.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:27:29.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:27:29.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-17T06:27:29.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:27:29.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051586 2026-02-17T06:27:29.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:27:29.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:27:29.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051586 2026-02-17T06:27:29.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:27:29.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T06:27:29.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-17T06:27:29.321 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 124554051586 2026-02-17T06:27:29.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:27:29.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T06:27:29.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:27:29.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:27:29.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:27:29.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:27:29.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:27:29.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:27:29.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:27:29.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:27:29.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:27:29.666 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:27:29.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:27:29.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:27:29.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:27:29.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:27:29.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:27:30.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:27:30.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:27:30.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:27:30.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:781: _scrub_snaps_multi: local pgid=1.0 2026-02-17T06:27:30.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:782: _scrub_snaps_multi: pg_scrub 1.0 2026-02-17T06:27:30.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T06:27:30.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T06:27:30.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T06:27:30.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:27:30.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:27:30.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:27:30.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:27:30.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:27:30.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:27:30.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:27:30.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:27:30.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:27:30.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:27:30.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:27:30.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:27:30.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:27:30.433 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:27:30.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:27:30.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:27:30.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:27:30.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149700 2026-02-17T06:27:30.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149700 2026-02-17T06:27:30.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149700' 2026-02-17T06:27:30.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:27:30.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:27:30.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051587 2026-02-17T06:27:30.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051587 2026-02-17T06:27:30.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-111669149700 1-124554051587' 2026-02-17T06:27:30.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:27:30.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-111669149700 2026-02-17T06:27:30.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:27:30.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:27:30.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-111669149700 2026-02-17T06:27:30.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:27:30.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149700 2026-02-17T06:27:30.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 111669149700' 2026-02-17T06:27:30.591 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 111669149700 2026-02-17T06:27:30.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:27:30.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149700 2026-02-17T06:27:30.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:27:31.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:27:31.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:27:31.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-17T06:27:31.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:27:31.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051587 2026-02-17T06:27:31.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:27:31.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:27:31.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051587 2026-02-17T06:27:31.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:27:31.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051587 2026-02-17T06:27:31.916 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 124554051587 2026-02-17T06:27:31.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051587' 2026-02-17T06:27:31.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:27:32.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051587 -lt 124554051587 2026-02-17T06:27:32.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:27:32.076 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 1.0 loop 0 2026-02-17T06:27:32.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T06:27:32.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T06:27:32.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T06:27:32.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:27:32.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T06:27:32.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:27:32.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:27:32.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:27:32.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:32.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T06:27:32.452 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-17T06:27:32.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:27:32.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:27:32.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:26:29.400304+0000 '>' 2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:32.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:27:33.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:27:33.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:27:33.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:27:33.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:27:33.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:27:33.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:27:33.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:27:33.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:26:29.400304+0000 '>' 2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:33.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:27:34.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:27:34.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:27:34.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:27:34.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:27:34.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:27:34.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:27:34.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:27:34.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:26:29.400304+0000 '>' 2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:34.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:27:35.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:27:35.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:27:35.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:27:35.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:27:35.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:27:35.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:27:35.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:27:36.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:26:29.400304+0000 '>' 2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:36.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:27:37.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:27:37.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T06:27:37.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:27:37.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:27:37.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:27:37.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:27:37.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:27:37.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:27:32.469966+0000 '>' 2026-02-17T06:26:29.400304+0000 2026-02-17T06:27:37.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:27:37.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:37.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: wc -l 2026-02-17T06:27:37.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: test 16 -gt 3 2026-02-17T06:27:37.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:27:37.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: wc -l 2026-02-17T06:27:37.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: test 16 -gt 3 2026-02-17T06:27:37.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:789: _scrub_snaps_multi: rados list-inconsistent-pg test 2026-02-17T06:27:37.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: jq '. | length' td/osd-scrub-snaps/json 2026-02-17T06:27:37.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: test 1 = 1 2026-02-17T06:27:37.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-17T06:27:37.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: test 1.0 = 1.0 2026-02-17T06:27:37.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:795: _scrub_snaps_multi: rados list-inconsistent-obj 1.0 --format=json-pretty 2026-02-17T06:27:37.313 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:27:37.313 INFO:tasks.workunit.client.0.trial074.stdout: "epoch": 29, 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "inconsistents": [ 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "version": 5 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.314 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 1, 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.315 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'19", 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'5", 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4187.0:1", 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 5, 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:31.972316+0000", 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:31.972697+0000", 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.316 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xed75bc05", 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.317 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.318 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.319 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 2, 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "version": 20 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.320 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 2, 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "version": "18'41", 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "17'20", 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4235.0:1", 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 20, 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:32.895574+0000", 2026-02-17T06:27:37.321 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:32.895866+0000", 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.322 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.323 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.324 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 4, 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "version": 42 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch" 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:27:37.325 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 4, 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "version": "20'45", 2026-02-17T06:27:37.326 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "18'42", 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4271.0:1", 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 42, 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "size": 512, 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.285988+0000", 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.286429+0000", 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.327 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xeaeeb5b0", 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.328 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "size": 4608, 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "object_info": { 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj5", 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 4, 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1718170787, 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.329 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "version": "20'45", 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "18'42", 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4271.0:1", 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 42, 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "size": 512, 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.285988+0000", 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.286429+0000", 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.330 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xeaeeb5b0", 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.331 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: "size": 512 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.332 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj4", 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 7, 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "version": 4 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.333 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj4", 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": 7, 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2826278768, 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "version": "23'51", 2026-02-17T06:27:37.334 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'4", 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4184.0:1", 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 4, 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:31.944515+0000", 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:31.944909+0000", 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.335 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xed75bc05", 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.336 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.337 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj5", 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 7, 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: "version": 0 2026-02-17T06:27:37.338 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "missing", 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "info_missing" 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "info_missing" 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.339 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1792 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.340 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj1", 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "version": 18 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.341 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj1", 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1828249343, 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'18", 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'1", 2026-02-17T06:27:37.342 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4232.0:1", 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 18, 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:32.865128+0000", 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:32.866556+0000", 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.343 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.344 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: "missing" 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.345 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj10", 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "version": 32 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.346 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj10", 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 718195851, 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'32", 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'10", 2026-02-17T06:27:37.347 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4253.0:1", 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 32, 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.059773+0000", 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.060183+0000", 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.348 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.349 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "????", 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.350 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.351 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj11", 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "version": 34 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.352 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj11", 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 693400951, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'34", 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'11", 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4256.0:1", 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 34, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.087047+0000", 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.087406+0000", 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.353 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.354 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.355 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.356 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj13", 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "version": 38 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj13", 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2087409765, 2026-02-17T06:27:37.357 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'38", 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'13", 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4262.0:1", 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 38, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.144494+0000", 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.144837+0000", 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.358 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.359 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]" 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.360 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj14", 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "version": 40 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.361 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj14", 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2484217095, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'40", 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'14", 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4265.0:1", 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 40, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.174486+0000", 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.174930+0000", 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.362 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.363 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1033, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.364 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj15", 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.365 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "version": 15 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_corrupted" 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj15", 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 612772309, 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "version": "16'15", 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "0'0", 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4217.0:1", 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 15, 2026-02-17T06:27:37.366 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:32.256338+0000", 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:32.256682+0000", 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xed75bc05", 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.367 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_corrupted" 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": "Z2FyYmFnZQo=" 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.368 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj2", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "version": 56 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_missing" 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj2", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1058988552, 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "version": "23'56", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "20'48", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4307.0:1", 2026-02-17T06:27:37.369 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 56, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "size": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.843544+0000", 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.843944+0000", 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "whiteout", 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "dirty" 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xffffffff", 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.370 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_missing" 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "size": 0 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "size": 0, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 7, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 4, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: 4, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: 3, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: 2, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.371 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 7, 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1024, 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: 7, 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: 6, 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: 5 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj3", 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "version": 44 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch" 2026-02-17T06:27:37.372 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [ 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj3", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1643547569, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19'44", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "17'22", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4277.0:1", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 44, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "size": 768, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.380167+0000", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.380533+0000", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.373 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x37904fe2", 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "size_mismatch_info", 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "obj_size_info_mismatch" 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "size": 3840, 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "object_info": { 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj3", 2026-02-17T06:27:37.374 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1643547569, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19'44", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "17'22", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4277.0:1", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 44, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "size": 768, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.380167+0000", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.380533+0000", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0x37904fe2", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.375 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "size": 768 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj6", 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "version": 24 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.376 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj6", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2202164420, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'24", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'6", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4241.0:1", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 24, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:32.949530+0000", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:32.949917+0000", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.377 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.378 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj7", 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "version": 26 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj7", 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 1552453721, 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.379 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'26", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'7", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4244.0:1", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 26, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:32.979568+0000", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:32.979948+0000", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.380 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [] 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.381 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj8", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "version": 28 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj8", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 2381834917, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'28", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'8", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4247.0:1", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 28, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.008347+0000", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.008703+0000", 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.382 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 0, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.383 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "object": { 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "name": "obj9", 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "nspace": "", 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "locator": "", 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "snap": "head", 2026-02-17T06:27:37.384 INFO:tasks.workunit.client.0.trial074.stdout: "version": 30 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [ 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "snapset_inconsistency" 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "union_shard_errors": [], 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "selected_object_info": { 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "oid": { 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "oid": "obj9", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "key": "", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "snapid": -2, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "hash": 3833113727, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "max": 0, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "pool": 1, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "namespace": "" 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "version": "17'30", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "prior_version": "16'9", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "last_reqid": "client.4250.0:1", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "user_version": 30, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "mtime": "2026-02-17T06:26:33.038320+0000", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "local_mtime": "2026-02-17T06:26:33.038664+0000", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "lost": 0, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "flags": [ 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "dirty", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest" 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_seq": 0, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "truncate_size": 0, 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "data_digest": "0xef17951f", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "omap_digest": "0xffffffff", 2026-02-17T06:27:37.385 INFO:tasks.workunit.client.0.trial074.stdout: "expected_object_size": 0, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "expected_write_size": 0, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "alloc_hint_flags": 0, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "manifest": { 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "type": 0 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "watchers": {} 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "shards": [ 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 0, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "primary": false, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "size": "????", 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.386 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "osd": 1, 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "primary": true, 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "errors": [], 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "size": 256, 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "snapset": { 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "seq": 1, 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "clones": [ 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "snap": 1, 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "size": 1032, 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "overlap": "[]", 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "snaps": [ 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: 1 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:27:37.387 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:797: _scrub_snaps_multi: rados list-inconsistent-snapset 1.0 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:800: _scrub_snaps_multi: '[' replica = replica ']' 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:802: _scrub_snaps_multi: scruberrors=20 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:803: _scrub_snaps_multi: jq .inconsistents 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:803: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T06:27:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1067: _scrub_snaps_multi: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1068: _scrub_snaps_multi: test no = yes 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1073: _scrub_snaps_multi: test '' = yes 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: find td/osd-scrub-snaps 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: grep 'osd[^/]*\.pid' 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: pidfiles='td/osd-scrub-snaps/osd.1.pid 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:td/osd-scrub-snaps/osd.0.pid' 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1079: _scrub_snaps_multi: pids= 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.1.pid 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='506253 ' 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.0.pid 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='505638 ' 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1085: _scrub_snaps_multi: ERRORS=0 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1089: _scrub_snaps_multi: '[' replica = primary ']' 2026-02-17T06:27:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1111: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T06:27:37.517 INFO:tasks.workunit.client.0.trial074.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T06:27:37.518 INFO:tasks.workunit.client.0.trial074.stdout:1.0 36 0 0 0 0 24448 0 0 56 0 56 active+clean+inconsistent 2026-02-17T06:27:32.470034+0000 23'56 30:102 [1,0] 1 [1,0] 1 23'56 2026-02-17T06:27:32.469966+0000 0'0 2026-02-17T06:26:29.400304+0000 0 1 periodic scrub scheduled @ 2026-02-18T11:31:51.099303+0000 36 0 2026-02-17T06:27:37.518 INFO:tasks.workunit.client.0.trial074.stdout: 2026-02-17T06:27:37.518 INFO:tasks.workunit.client.0.trial074.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T06:27:37.518 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 506253 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 505638 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1122: _scrub_snaps_multi: kill_daemons td/osd-scrub-snaps 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:27:37.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:27:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:27:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1124: _scrub_snaps_multi: declare -a err_strings 2026-02-17T06:27:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1125: _scrub_snaps_multi: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' 2026-02-17T06:27:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1126: _scrub_snaps_multi: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' 2026-02-17T06:27:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1127: _scrub_snaps_multi: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' 2026-02-17T06:27:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1128: _scrub_snaps_multi: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' 2026-02-17T06:27:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1129: _scrub_snaps_multi: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' 2026-02-17T06:27:45.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1130: _scrub_snaps_multi: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' 2026-02-17T06:27:45.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1131: _scrub_snaps_multi: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' 2026-02-17T06:27:45.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1132: _scrub_snaps_multi: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' 2026-02-17T06:27:45.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T06:27:45.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' td/osd-scrub-snaps/osd.1.log 2026-02-17T06:27:45.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1144: _scrub_snaps_multi: declare -a rep_err_strings 2026-02-17T06:27:45.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: eval echo '$replica' 2026-02-17T06:27:45.864 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: echo 0 2026-02-17T06:27:45.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: osd=0 2026-02-17T06:27:45.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1146: _scrub_snaps_multi: rep_err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-17T06:27:45.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1147: _scrub_snaps_multi: for err_string in "${rep_err_strings[@]}" 2026-02-17T06:27:45.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1149: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.0.log 2026-02-17T06:27:45.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1156: _scrub_snaps_multi: '[' 0 '!=' 0 ']' 2026-02-17T06:27:45.867 INFO:tasks.workunit.client.0.trial074.stdout:TEST PASSED 2026-02-17T06:27:45.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1162: _scrub_snaps_multi: echo 'TEST PASSED' 2026-02-17T06:27:45.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1163: _scrub_snaps_multi: return 0 2026-02-17T06:27:45.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1171: TEST_scrub_snaps_replica: err=0 2026-02-17T06:27:45.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1172: TEST_scrub_snaps_replica: CEPH_ARGS='--fsid=186de67e-86bc-45e7-88b5-9b85e0a7ff16 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T06:27:45.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1173: TEST_scrub_snaps_replica: return 0 2026-02-17T06:27:45.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-17T06:27:45.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T06:27:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:27:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T06:27:45.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:27:45.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:27:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:27:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:27:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:27:45.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:27:45.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:27:45.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:27:45.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:27:45.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:27:45.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:27:45.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:27:45.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:27:45.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:27:45.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:27:45.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:27:45.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:27:45.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:27:45.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T06:27:45.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:27:45.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:45.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:27:45.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.451782 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-snaps 0 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:27:45.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:27:45.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:27:45.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:27:45.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:27:45.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:27:45.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:27:45.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:27:45.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:27:45.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:27:45.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:27:45.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:27:45.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:27:45.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:27:45.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:27:45.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:27:45.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:27:45.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T06:27:45.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T06:27:45.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:27:45.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:45.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.451782 2026-02-17T06:27:45.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.451782 2026-02-17T06:27:45.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:27:45.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:27:45.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T06:27:45.911 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T06:27:45.911 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T06:27:45.960 INFO:tasks.workunit:Running workunit scrub/osd-scrub-test.sh... 2026-02-17T06:27:45.960 DEBUG:teuthology.orchestra.run.trial074:workunit test scrub/osd-scrub-test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh 2026-02-17T06:27:46.013 INFO:tasks.workunit.client.0.trial074.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T06:27:46.017 INFO:tasks.workunit.client.0.trial074.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T06:27:46.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T06:27:46.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-test 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:21: run: local dir=td/osd-scrub-test 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:22: run: shift 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:24: run: export CEPH_MON=127.0.0.1:7138 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:24: run: CEPH_MON=127.0.0.1:7138 2026-02-17T06:27:46.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:25: run: export CEPH_ARGS 2026-02-17T06:27:46.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:26: run: uuidgen 2026-02-17T06:27:46.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:26: run: CEPH_ARGS+='--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none ' 2026-02-17T06:27:46.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7138 ' 2026-02-17T06:27:46.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:29: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T06:27:46.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: set 2026-02-17T06:27:46.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T06:27:46.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: local 'funcs=TEST_deep_scrub_abort 2026-02-17T06:27:46.021 INFO:tasks.workunit.client.0.trial074.stderr:TEST_dump_scrub_schedule 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:TEST_interval_changes 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:TEST_just_deep_scrubs 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:TEST_pg_dump_objects_scrubbed 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:TEST_scrub_abort 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:TEST_scrub_extended_sleep 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:TEST_scrub_permit_time 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:TEST_scrub_test' 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:27:46.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_deep_scrub_abort -------------------' 2026-02-17T06:27:46.023 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_deep_scrub_abort ------------------- 2026-02-17T06:27:46.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:27:46.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:27:46.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:27:46.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:27:46.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:27:46.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:27:46.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:27:46.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:27:46.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:27:46.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:27:46.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:27:46.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:27:46.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:27:46.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:27:46.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:27:46.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:27:46.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:27:46.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:27:46.032 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:27:46.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:46.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:46.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:27:46.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:27:46.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:27:46.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:27:46.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:27:46.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:46.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:46.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:27:46.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:27:46.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T06:27:46.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T06:27:46.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:27:46.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:27:46.038 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_deep_scrub_abort ----------------------- 2026-02-17T06:27:46.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_deep_scrub_abort -----------------------' 2026-02-17T06:27:46.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_deep_scrub_abort td/osd-scrub-test 2026-02-17T06:27:46.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:414: TEST_deep_scrub_abort: local dir=td/osd-scrub-test 2026-02-17T06:27:46.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:415: TEST_deep_scrub_abort: _scrub_abort td/osd-scrub-test deep-scrub 2026-02-17T06:27:46.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:298: _scrub_abort: local dir=td/osd-scrub-test 2026-02-17T06:27:46.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:299: _scrub_abort: local poolname=test 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:300: _scrub_abort: local OSDS=3 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:301: _scrub_abort: local objects=1000 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:302: _scrub_abort: local type=deep-scrub 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:304: _scrub_abort: TESTDATA=testdata.507902 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:305: _scrub_abort: test deep-scrub = scrub 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:310: _scrub_abort: stopscrub=nodeep-scrub 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: check=nodeep_scrub 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:314: _scrub_abort: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:27:46.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:27:46.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:27:46.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T06:27:46.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:27:46.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:27:46.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:27:46.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:27:46.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:46.062 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:46.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:27:46.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T06:27:46.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:27:46.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:27:46.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:27:46.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:27:46.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:27:46.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:27:46.092 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:27:46.092 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:27:46.092 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:27:46.092 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:27:46.092 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:46.092 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:46.092 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:27:46.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:27:46.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:27:46.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:46.147 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:46.148 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:27:46.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:27:46.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:27:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:315: _scrub_abort: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:27:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:27:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:27:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:27:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:27:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:27:46.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:27:46.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:27:46.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:27:46.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:27:46.293 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:27:46.293 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:46.293 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:46.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:27:46.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:27:46.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:27:46.312 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: expr 3 - 1 2026-02-17T06:27:46.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: seq 0 2 2026-02-17T06:27:46.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:27:46.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 0 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:27:46.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:27:46.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:27:46.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:27:46.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:27:46.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:27:46.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:27:46.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c488c838-c082-45c0-a75b-eb0dcbb1308e 2026-02-17T06:27:46.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 c488c838-c082-45c0-a75b-eb0dcbb1308e' 2026-02-17T06:27:46.321 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 c488c838-c082-45c0-a75b-eb0dcbb1308e 2026-02-17T06:27:46.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:27:46.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBiCpRp6NdPExAAsVb49MXVPfxCLIxErbIyTw== 2026-02-17T06:27:46.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBiCpRp6NdPExAAsVb49MXVPfxCLIxErbIyTw=="}' 2026-02-17T06:27:46.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c488c838-c082-45c0-a75b-eb0dcbb1308e -i td/osd-scrub-test/0/new.json 2026-02-17T06:27:46.418 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:27:46.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:27:46.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBiCpRp6NdPExAAsVb49MXVPfxCLIxErbIyTw== --osd-uuid c488c838-c082-45c0-a75b-eb0dcbb1308e 2026-02-17T06:27:46.443 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:46.433+0000 7f3268b8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:46.444 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:46.437+0000 7f3268b8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:46.445 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:46.437+0000 7f3268b8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:46.445 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:46.437+0000 7f3268b8a8c0 -1 bdev(0x55798928fc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:27:46.446 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:46.437+0000 7f3268b8a8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:27:49.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:27:49.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:27:49.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:27:49.494 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:27:49.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:27:49.685 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:27:49.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:27:49.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:27:49.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:27:49.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:27:49.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:27:49.709 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:49.701+0000 7fd16bee48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:49.710 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:49.701+0000 7fd16bee48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:49.712 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:49.701+0000 7fd16bee48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:49.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:27:49.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:27:49.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:27:49.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:27:49.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:27:49.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:49.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:27:49.839 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:27:49.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:49.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:49.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:50.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:50.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:50.990 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:27:50.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:27:50.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:50.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:51.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:51.266 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:51.257+0000 7fd16bee48c0 -1 Falling back to public interface 2026-02-17T06:27:52.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:52.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:52.138 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:27:52.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:27:52.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:52.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:52.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:53.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:53.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:53.289 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:27:53.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:27:53.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:53.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:53.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:54.284 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:54.277+0000 7fd16bee48c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:27:54.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:54.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:54.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:27:54.440 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:27:54.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:54.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:27:54.593 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4135990047,v1:127.0.0.1:6803/4135990047] [v2:127.0.0.1:6804/4135990047,v1:127.0.0.1:6805/4135990047] exists,up c488c838-c082-45c0-a75b-eb0dcbb1308e 2026-02-17T06:27:54.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:27:54.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:27:54.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:27:54.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:27:54.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 1 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:27:54.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:27:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:27:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:27:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:27:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:27:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:27:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:27:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:27:54.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:27:54.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:27:54.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:27:54.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:27:54.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:27:54.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:27:54.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:27:54.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:27:54.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:27:54.601 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:27:54.601 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:27:54.601 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:27:54.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:27:54.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:27:54.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:27:54.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:27:54.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:27:54.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:27:54.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b0167f57-8e1c-4a05-8b83-c624521dfa24 2026-02-17T06:27:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 b0167f57-8e1c-4a05-8b83-c624521dfa24' 2026-02-17T06:27:54.604 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 b0167f57-8e1c-4a05-8b83-c624521dfa24 2026-02-17T06:27:54.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:27:54.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBqCpRp5/YNJBAAKf/S2SeAdHnRTddJFlLx/w== 2026-02-17T06:27:54.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBqCpRp5/YNJBAAKf/S2SeAdHnRTddJFlLx/w=="}' 2026-02-17T06:27:54.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b0167f57-8e1c-4a05-8b83-c624521dfa24 -i td/osd-scrub-test/1/new.json 2026-02-17T06:27:54.753 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:27:54.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:27:54.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBqCpRp5/YNJBAAKf/S2SeAdHnRTddJFlLx/w== --osd-uuid b0167f57-8e1c-4a05-8b83-c624521dfa24 2026-02-17T06:27:54.780 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:54.773+0000 7f9a629168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:54.782 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:54.773+0000 7f9a629168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:54.783 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:54.773+0000 7f9a629168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:54.784 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:54.773+0000 7f9a629168c0 -1 bdev(0x56430de8fc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:27:54.784 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:54.773+0000 7f9a629168c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:27:57.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:27:57.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:27:57.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:27:57.086 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:27:57.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:27:57.274 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:27:57.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:27:57.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:27:57.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:27:57.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:27:57.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:27:57.298 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:57.289+0000 7ff5354d68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:57.300 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:57.289+0000 7ff5354d68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:57.301 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:57.293+0000 7ff5354d68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:27:57.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:27:57.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:27:57.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:27:57.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:27:57.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:27:57.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:57.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:27:57.429 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:27:57.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:57.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:57.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:58.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:58.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:58.580 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:27:58.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:27:58.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:58.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:58.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:27:58.802 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:27:58.793+0000 7ff5354d68c0 -1 Falling back to public interface 2026-02-17T06:27:59.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:27:59.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:27:59.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:27:59.726 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:27:59.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:27:59.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:27:59.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:28:00.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:28:00.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:28:00.878 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:28:00.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:28:00.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:28:00.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:28:01.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:28:01.776 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:01.769+0000 7ff5354d68c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:28:02.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:28:02.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:28:02.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:28:02.027 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:28:02.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:28:02.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:28:02.176 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3533245772,v1:127.0.0.1:6811/3533245772] [v2:127.0.0.1:6812/3533245772,v1:127.0.0.1:6813/3533245772] exists,up b0167f57-8e1c-4a05-8b83-c624521dfa24 2026-02-17T06:28:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:28:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:28:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:28:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:28:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 2 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:28:02.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:28:02.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:28:02.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T06:28:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:28:02.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:28:02.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e8997e67-09ba-4583-a8a9-999d3bfc3fb0 2026-02-17T06:28:02.183 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 e8997e67-09ba-4583-a8a9-999d3bfc3fb0 2026-02-17T06:28:02.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 e8997e67-09ba-4583-a8a9-999d3bfc3fb0' 2026-02-17T06:28:02.184 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:28:02.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQByCpRpiQI8CxAAv13zJFkHZn0AEpHyUAbmGw== 2026-02-17T06:28:02.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQByCpRpiQI8CxAAv13zJFkHZn0AEpHyUAbmGw=="}' 2026-02-17T06:28:02.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e8997e67-09ba-4583-a8a9-999d3bfc3fb0 -i td/osd-scrub-test/2/new.json 2026-02-17T06:28:02.337 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:28:02.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T06:28:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQByCpRpiQI8CxAAv13zJFkHZn0AEpHyUAbmGw== --osd-uuid e8997e67-09ba-4583-a8a9-999d3bfc3fb0 2026-02-17T06:28:02.363 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:02.353+0000 7f13894648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:28:02.364 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:02.357+0000 7f13894648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:28:02.365 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:02.357+0000 7f13894648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:28:02.366 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:02.357+0000 7f13894648c0 -1 bdev(0x558ec6929c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:28:02.366 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:02.357+0000 7f13894648c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:28:04.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T06:28:04.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:28:04.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:28:04.618 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:28:04.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:28:04.809 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:28:04.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:28:04.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:28:04.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:28:04.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:28:04.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:28:04.833 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:04.825+0000 7fe6096268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:28:04.835 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:04.825+0000 7fe6096268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:28:04.836 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:04.825+0000 7fe6096268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:28:04.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:28:04.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:28:04.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:28:04.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:28:04.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:28:04.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:28:04.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:28:04.966 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:28:04.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:28:04.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:28:05.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:28:06.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:28:06.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:28:06.114 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:28:06.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:28:06.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:28:06.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:28:06.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:28:06.326 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:06.317+0000 7fe6096268c0 -1 Falling back to public interface 2026-02-17T06:28:07.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:28:07.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:28:07.261 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:28:07.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:28:07.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:28:07.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:28:07.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:28:08.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:28:08.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:28:08.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:28:08.408 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:28:08.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:28:08.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:28:08.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:28:09.376 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:28:09.372+0000 7fe6096268c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:28:09.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:28:09.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:28:09.557 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:28:09.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:28:09.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:28:09.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:28:09.705 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3650311374,v1:127.0.0.1:6819/3650311374] [v2:127.0.0.1:6820/3650311374,v1:127.0.0.1:6821/3650311374] exists,up e8997e67-09ba-4583-a8a9-999d3bfc3fb0 2026-02-17T06:28:09.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:28:09.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:28:09.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:28:09.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: create_pool test 1 1 2026-02-17T06:28:09.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:28:09.875 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:28:09.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:28:10.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:331: _scrub_abort: wait_for_clean 2026-02-17T06:28:10.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:28:10.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:28:10.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:28:10.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:28:10.887 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:28:10.888 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:28:10.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:28:10.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:28:10.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:28:10.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:28:10.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:28:10.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:28:10.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:28:10.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:10.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:11.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:11.146 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:11.146 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:11.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:11.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:11.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:11.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T06:28:11.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T06:28:11.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T06:28:11.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:11.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:11.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T06:28:11.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T06:28:11.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-38654705674' 2026-02-17T06:28:11.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:11.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:11.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T06:28:11.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T06:28:11.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-38654705674 2-60129542146' 2026-02-17T06:28:11.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:11.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T06:28:11.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:11.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:11.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T06:28:11.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:11.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T06:28:11.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T06:28:11.359 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836497 2026-02-17T06:28:11.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:11.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T06:28:11.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:12.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:12.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:12.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836497 2026-02-17T06:28:12.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:12.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T06:28:12.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:12.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:12.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T06:28:12.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T06:28:12.675 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705674 2026-02-17T06:28:12.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T06:28:12.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:12.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705674 2026-02-17T06:28:12.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:12.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T06:28:12.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:12.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:12.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T06:28:12.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:12.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T06:28:12.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T06:28:12.841 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542146 2026-02-17T06:28:12.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:12.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542146 2026-02-17T06:28:12.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:28:12.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:28:12.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:28:13.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:28:13.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:28:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:28:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:28:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:28:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:28:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:28:13.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:28:13.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:28:13.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:28:13.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:28:13.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:28:13.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:28:13.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:28:13.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:28:13.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: ceph osd dump 2026-02-17T06:28:13.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: grep '^pool.*['\'']test['\'']' 2026-02-17T06:28:13.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: awk '{ print $2 }' 2026-02-17T06:28:13.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: poolid=1 2026-02-17T06:28:13.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:334: _scrub_abort: dd if=/dev/urandom of=testdata.507902 bs=1032 count=1 2026-02-17T06:28:13.645 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:28:13.645 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:28:13.645 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.3943e-05 s, 30.4 MB/s 2026-02-17T06:28:13.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: seq 1 1000 2026-02-17T06:28:13.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1 testdata.507902 2026-02-17T06:28:13.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj2 testdata.507902 2026-02-17T06:28:13.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj3 testdata.507902 2026-02-17T06:28:13.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj4 testdata.507902 2026-02-17T06:28:13.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj5 testdata.507902 2026-02-17T06:28:13.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj6 testdata.507902 2026-02-17T06:28:13.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj7 testdata.507902 2026-02-17T06:28:13.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj8 testdata.507902 2026-02-17T06:28:13.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj9 testdata.507902 2026-02-17T06:28:13.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj10 testdata.507902 2026-02-17T06:28:13.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj11 testdata.507902 2026-02-17T06:28:13.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj12 testdata.507902 2026-02-17T06:28:13.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj13 testdata.507902 2026-02-17T06:28:13.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj14 testdata.507902 2026-02-17T06:28:13.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj15 testdata.507902 2026-02-17T06:28:13.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:13.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj16 testdata.507902 2026-02-17T06:28:14.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj17 testdata.507902 2026-02-17T06:28:14.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj18 testdata.507902 2026-02-17T06:28:14.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj19 testdata.507902 2026-02-17T06:28:14.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj20 testdata.507902 2026-02-17T06:28:14.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj21 testdata.507902 2026-02-17T06:28:14.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj22 testdata.507902 2026-02-17T06:28:14.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj23 testdata.507902 2026-02-17T06:28:14.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj24 testdata.507902 2026-02-17T06:28:14.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj25 testdata.507902 2026-02-17T06:28:14.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj26 testdata.507902 2026-02-17T06:28:14.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj27 testdata.507902 2026-02-17T06:28:14.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj28 testdata.507902 2026-02-17T06:28:14.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj29 testdata.507902 2026-02-17T06:28:14.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj30 testdata.507902 2026-02-17T06:28:14.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj31 testdata.507902 2026-02-17T06:28:14.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj32 testdata.507902 2026-02-17T06:28:14.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj33 testdata.507902 2026-02-17T06:28:14.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj34 testdata.507902 2026-02-17T06:28:14.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj35 testdata.507902 2026-02-17T06:28:14.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj36 testdata.507902 2026-02-17T06:28:14.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj37 testdata.507902 2026-02-17T06:28:14.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj38 testdata.507902 2026-02-17T06:28:14.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj39 testdata.507902 2026-02-17T06:28:14.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj40 testdata.507902 2026-02-17T06:28:14.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj41 testdata.507902 2026-02-17T06:28:14.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj42 testdata.507902 2026-02-17T06:28:14.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj43 testdata.507902 2026-02-17T06:28:14.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj44 testdata.507902 2026-02-17T06:28:14.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj45 testdata.507902 2026-02-17T06:28:14.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj46 testdata.507902 2026-02-17T06:28:14.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj47 testdata.507902 2026-02-17T06:28:14.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj48 testdata.507902 2026-02-17T06:28:14.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj49 testdata.507902 2026-02-17T06:28:14.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj50 testdata.507902 2026-02-17T06:28:14.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj51 testdata.507902 2026-02-17T06:28:14.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj52 testdata.507902 2026-02-17T06:28:14.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj53 testdata.507902 2026-02-17T06:28:14.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj54 testdata.507902 2026-02-17T06:28:14.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj55 testdata.507902 2026-02-17T06:28:14.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj56 testdata.507902 2026-02-17T06:28:14.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj57 testdata.507902 2026-02-17T06:28:14.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:14.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj58 testdata.507902 2026-02-17T06:28:15.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj59 testdata.507902 2026-02-17T06:28:15.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj60 testdata.507902 2026-02-17T06:28:15.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj61 testdata.507902 2026-02-17T06:28:15.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj62 testdata.507902 2026-02-17T06:28:15.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj63 testdata.507902 2026-02-17T06:28:15.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj64 testdata.507902 2026-02-17T06:28:15.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj65 testdata.507902 2026-02-17T06:28:15.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj66 testdata.507902 2026-02-17T06:28:15.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj67 testdata.507902 2026-02-17T06:28:15.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj68 testdata.507902 2026-02-17T06:28:15.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj69 testdata.507902 2026-02-17T06:28:15.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj70 testdata.507902 2026-02-17T06:28:15.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj71 testdata.507902 2026-02-17T06:28:15.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj72 testdata.507902 2026-02-17T06:28:15.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj73 testdata.507902 2026-02-17T06:28:15.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj74 testdata.507902 2026-02-17T06:28:15.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj75 testdata.507902 2026-02-17T06:28:15.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj76 testdata.507902 2026-02-17T06:28:15.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj77 testdata.507902 2026-02-17T06:28:15.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj78 testdata.507902 2026-02-17T06:28:15.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj79 testdata.507902 2026-02-17T06:28:15.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj80 testdata.507902 2026-02-17T06:28:15.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj81 testdata.507902 2026-02-17T06:28:15.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj82 testdata.507902 2026-02-17T06:28:15.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj83 testdata.507902 2026-02-17T06:28:15.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj84 testdata.507902 2026-02-17T06:28:15.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj85 testdata.507902 2026-02-17T06:28:15.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj86 testdata.507902 2026-02-17T06:28:15.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj87 testdata.507902 2026-02-17T06:28:15.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj88 testdata.507902 2026-02-17T06:28:15.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj89 testdata.507902 2026-02-17T06:28:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj90 testdata.507902 2026-02-17T06:28:15.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj91 testdata.507902 2026-02-17T06:28:15.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj92 testdata.507902 2026-02-17T06:28:15.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj93 testdata.507902 2026-02-17T06:28:15.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj94 testdata.507902 2026-02-17T06:28:15.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj95 testdata.507902 2026-02-17T06:28:15.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj96 testdata.507902 2026-02-17T06:28:15.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj97 testdata.507902 2026-02-17T06:28:15.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj98 testdata.507902 2026-02-17T06:28:15.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj99 testdata.507902 2026-02-17T06:28:15.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj100 testdata.507902 2026-02-17T06:28:15.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:15.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj101 testdata.507902 2026-02-17T06:28:16.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj102 testdata.507902 2026-02-17T06:28:16.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj103 testdata.507902 2026-02-17T06:28:16.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj104 testdata.507902 2026-02-17T06:28:16.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj105 testdata.507902 2026-02-17T06:28:16.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj106 testdata.507902 2026-02-17T06:28:16.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj107 testdata.507902 2026-02-17T06:28:16.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj108 testdata.507902 2026-02-17T06:28:16.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj109 testdata.507902 2026-02-17T06:28:16.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj110 testdata.507902 2026-02-17T06:28:16.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj111 testdata.507902 2026-02-17T06:28:16.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj112 testdata.507902 2026-02-17T06:28:16.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj113 testdata.507902 2026-02-17T06:28:16.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj114 testdata.507902 2026-02-17T06:28:16.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj115 testdata.507902 2026-02-17T06:28:16.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj116 testdata.507902 2026-02-17T06:28:16.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj117 testdata.507902 2026-02-17T06:28:16.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj118 testdata.507902 2026-02-17T06:28:16.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj119 testdata.507902 2026-02-17T06:28:16.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj120 testdata.507902 2026-02-17T06:28:16.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj121 testdata.507902 2026-02-17T06:28:16.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj122 testdata.507902 2026-02-17T06:28:16.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj123 testdata.507902 2026-02-17T06:28:16.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj124 testdata.507902 2026-02-17T06:28:16.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj125 testdata.507902 2026-02-17T06:28:16.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj126 testdata.507902 2026-02-17T06:28:16.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj127 testdata.507902 2026-02-17T06:28:16.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj128 testdata.507902 2026-02-17T06:28:16.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj129 testdata.507902 2026-02-17T06:28:16.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj130 testdata.507902 2026-02-17T06:28:16.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj131 testdata.507902 2026-02-17T06:28:16.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj132 testdata.507902 2026-02-17T06:28:16.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj133 testdata.507902 2026-02-17T06:28:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj134 testdata.507902 2026-02-17T06:28:16.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj135 testdata.507902 2026-02-17T06:28:16.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj136 testdata.507902 2026-02-17T06:28:16.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj137 testdata.507902 2026-02-17T06:28:16.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj138 testdata.507902 2026-02-17T06:28:16.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj139 testdata.507902 2026-02-17T06:28:16.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj140 testdata.507902 2026-02-17T06:28:16.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj141 testdata.507902 2026-02-17T06:28:16.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj142 testdata.507902 2026-02-17T06:28:16.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj143 testdata.507902 2026-02-17T06:28:16.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:16.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj144 testdata.507902 2026-02-17T06:28:17.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj145 testdata.507902 2026-02-17T06:28:17.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj146 testdata.507902 2026-02-17T06:28:17.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj147 testdata.507902 2026-02-17T06:28:17.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj148 testdata.507902 2026-02-17T06:28:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj149 testdata.507902 2026-02-17T06:28:17.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj150 testdata.507902 2026-02-17T06:28:17.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj151 testdata.507902 2026-02-17T06:28:17.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj152 testdata.507902 2026-02-17T06:28:17.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj153 testdata.507902 2026-02-17T06:28:17.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj154 testdata.507902 2026-02-17T06:28:17.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj155 testdata.507902 2026-02-17T06:28:17.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj156 testdata.507902 2026-02-17T06:28:17.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj157 testdata.507902 2026-02-17T06:28:17.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj158 testdata.507902 2026-02-17T06:28:17.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj159 testdata.507902 2026-02-17T06:28:17.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj160 testdata.507902 2026-02-17T06:28:17.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj161 testdata.507902 2026-02-17T06:28:17.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj162 testdata.507902 2026-02-17T06:28:17.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj163 testdata.507902 2026-02-17T06:28:17.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj164 testdata.507902 2026-02-17T06:28:17.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj165 testdata.507902 2026-02-17T06:28:17.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj166 testdata.507902 2026-02-17T06:28:17.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj167 testdata.507902 2026-02-17T06:28:17.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj168 testdata.507902 2026-02-17T06:28:17.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj169 testdata.507902 2026-02-17T06:28:17.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj170 testdata.507902 2026-02-17T06:28:17.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj171 testdata.507902 2026-02-17T06:28:17.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj172 testdata.507902 2026-02-17T06:28:17.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj173 testdata.507902 2026-02-17T06:28:17.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj174 testdata.507902 2026-02-17T06:28:17.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj175 testdata.507902 2026-02-17T06:28:17.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj176 testdata.507902 2026-02-17T06:28:17.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj177 testdata.507902 2026-02-17T06:28:17.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj178 testdata.507902 2026-02-17T06:28:17.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj179 testdata.507902 2026-02-17T06:28:17.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj180 testdata.507902 2026-02-17T06:28:17.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj181 testdata.507902 2026-02-17T06:28:17.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj182 testdata.507902 2026-02-17T06:28:17.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj183 testdata.507902 2026-02-17T06:28:17.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj184 testdata.507902 2026-02-17T06:28:17.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj185 testdata.507902 2026-02-17T06:28:17.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:17.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj186 testdata.507902 2026-02-17T06:28:18.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj187 testdata.507902 2026-02-17T06:28:18.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj188 testdata.507902 2026-02-17T06:28:18.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj189 testdata.507902 2026-02-17T06:28:18.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj190 testdata.507902 2026-02-17T06:28:18.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj191 testdata.507902 2026-02-17T06:28:18.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj192 testdata.507902 2026-02-17T06:28:18.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj193 testdata.507902 2026-02-17T06:28:18.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj194 testdata.507902 2026-02-17T06:28:18.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj195 testdata.507902 2026-02-17T06:28:18.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj196 testdata.507902 2026-02-17T06:28:18.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj197 testdata.507902 2026-02-17T06:28:18.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj198 testdata.507902 2026-02-17T06:28:18.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj199 testdata.507902 2026-02-17T06:28:18.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj200 testdata.507902 2026-02-17T06:28:18.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj201 testdata.507902 2026-02-17T06:28:18.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj202 testdata.507902 2026-02-17T06:28:18.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj203 testdata.507902 2026-02-17T06:28:18.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj204 testdata.507902 2026-02-17T06:28:18.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj205 testdata.507902 2026-02-17T06:28:18.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj206 testdata.507902 2026-02-17T06:28:18.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj207 testdata.507902 2026-02-17T06:28:18.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj208 testdata.507902 2026-02-17T06:28:18.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj209 testdata.507902 2026-02-17T06:28:18.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj210 testdata.507902 2026-02-17T06:28:18.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj211 testdata.507902 2026-02-17T06:28:18.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj212 testdata.507902 2026-02-17T06:28:18.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj213 testdata.507902 2026-02-17T06:28:18.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj214 testdata.507902 2026-02-17T06:28:18.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj215 testdata.507902 2026-02-17T06:28:18.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj216 testdata.507902 2026-02-17T06:28:18.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj217 testdata.507902 2026-02-17T06:28:18.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj218 testdata.507902 2026-02-17T06:28:18.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj219 testdata.507902 2026-02-17T06:28:18.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj220 testdata.507902 2026-02-17T06:28:18.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj221 testdata.507902 2026-02-17T06:28:18.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj222 testdata.507902 2026-02-17T06:28:18.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj223 testdata.507902 2026-02-17T06:28:18.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj224 testdata.507902 2026-02-17T06:28:18.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj225 testdata.507902 2026-02-17T06:28:18.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj226 testdata.507902 2026-02-17T06:28:18.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj227 testdata.507902 2026-02-17T06:28:18.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj228 testdata.507902 2026-02-17T06:28:18.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj229 testdata.507902 2026-02-17T06:28:18.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj230 testdata.507902 2026-02-17T06:28:18.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj231 testdata.507902 2026-02-17T06:28:18.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:18.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj232 testdata.507902 2026-02-17T06:28:19.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj233 testdata.507902 2026-02-17T06:28:19.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj234 testdata.507902 2026-02-17T06:28:19.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj235 testdata.507902 2026-02-17T06:28:19.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj236 testdata.507902 2026-02-17T06:28:19.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj237 testdata.507902 2026-02-17T06:28:19.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj238 testdata.507902 2026-02-17T06:28:19.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj239 testdata.507902 2026-02-17T06:28:19.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj240 testdata.507902 2026-02-17T06:28:19.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj241 testdata.507902 2026-02-17T06:28:19.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj242 testdata.507902 2026-02-17T06:28:19.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj243 testdata.507902 2026-02-17T06:28:19.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj244 testdata.507902 2026-02-17T06:28:19.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj245 testdata.507902 2026-02-17T06:28:19.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj246 testdata.507902 2026-02-17T06:28:19.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj247 testdata.507902 2026-02-17T06:28:19.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj248 testdata.507902 2026-02-17T06:28:19.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj249 testdata.507902 2026-02-17T06:28:19.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj250 testdata.507902 2026-02-17T06:28:19.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj251 testdata.507902 2026-02-17T06:28:19.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj252 testdata.507902 2026-02-17T06:28:19.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj253 testdata.507902 2026-02-17T06:28:19.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj254 testdata.507902 2026-02-17T06:28:19.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj255 testdata.507902 2026-02-17T06:28:19.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj256 testdata.507902 2026-02-17T06:28:19.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj257 testdata.507902 2026-02-17T06:28:19.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj258 testdata.507902 2026-02-17T06:28:19.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj259 testdata.507902 2026-02-17T06:28:19.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj260 testdata.507902 2026-02-17T06:28:19.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj261 testdata.507902 2026-02-17T06:28:19.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj262 testdata.507902 2026-02-17T06:28:19.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj263 testdata.507902 2026-02-17T06:28:19.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj264 testdata.507902 2026-02-17T06:28:19.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj265 testdata.507902 2026-02-17T06:28:19.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj266 testdata.507902 2026-02-17T06:28:19.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj267 testdata.507902 2026-02-17T06:28:19.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj268 testdata.507902 2026-02-17T06:28:19.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj269 testdata.507902 2026-02-17T06:28:19.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj270 testdata.507902 2026-02-17T06:28:19.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj271 testdata.507902 2026-02-17T06:28:19.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj272 testdata.507902 2026-02-17T06:28:19.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj273 testdata.507902 2026-02-17T06:28:19.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj274 testdata.507902 2026-02-17T06:28:19.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj275 testdata.507902 2026-02-17T06:28:19.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:19.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj276 testdata.507902 2026-02-17T06:28:20.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj277 testdata.507902 2026-02-17T06:28:20.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj278 testdata.507902 2026-02-17T06:28:20.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj279 testdata.507902 2026-02-17T06:28:20.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj280 testdata.507902 2026-02-17T06:28:20.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj281 testdata.507902 2026-02-17T06:28:20.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj282 testdata.507902 2026-02-17T06:28:20.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj283 testdata.507902 2026-02-17T06:28:20.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj284 testdata.507902 2026-02-17T06:28:20.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj285 testdata.507902 2026-02-17T06:28:20.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj286 testdata.507902 2026-02-17T06:28:20.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj287 testdata.507902 2026-02-17T06:28:20.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj288 testdata.507902 2026-02-17T06:28:20.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj289 testdata.507902 2026-02-17T06:28:20.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj290 testdata.507902 2026-02-17T06:28:20.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj291 testdata.507902 2026-02-17T06:28:20.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj292 testdata.507902 2026-02-17T06:28:20.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj293 testdata.507902 2026-02-17T06:28:20.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj294 testdata.507902 2026-02-17T06:28:20.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj295 testdata.507902 2026-02-17T06:28:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj296 testdata.507902 2026-02-17T06:28:20.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj297 testdata.507902 2026-02-17T06:28:20.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj298 testdata.507902 2026-02-17T06:28:20.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj299 testdata.507902 2026-02-17T06:28:20.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj300 testdata.507902 2026-02-17T06:28:20.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj301 testdata.507902 2026-02-17T06:28:20.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj302 testdata.507902 2026-02-17T06:28:20.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj303 testdata.507902 2026-02-17T06:28:20.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj304 testdata.507902 2026-02-17T06:28:20.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj305 testdata.507902 2026-02-17T06:28:20.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj306 testdata.507902 2026-02-17T06:28:20.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj307 testdata.507902 2026-02-17T06:28:20.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj308 testdata.507902 2026-02-17T06:28:20.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj309 testdata.507902 2026-02-17T06:28:20.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj310 testdata.507902 2026-02-17T06:28:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj311 testdata.507902 2026-02-17T06:28:20.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj312 testdata.507902 2026-02-17T06:28:20.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj313 testdata.507902 2026-02-17T06:28:20.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj314 testdata.507902 2026-02-17T06:28:20.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj315 testdata.507902 2026-02-17T06:28:20.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj316 testdata.507902 2026-02-17T06:28:20.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj317 testdata.507902 2026-02-17T06:28:20.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj318 testdata.507902 2026-02-17T06:28:20.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:20.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj319 testdata.507902 2026-02-17T06:28:21.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj320 testdata.507902 2026-02-17T06:28:21.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj321 testdata.507902 2026-02-17T06:28:21.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj322 testdata.507902 2026-02-17T06:28:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj323 testdata.507902 2026-02-17T06:28:21.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj324 testdata.507902 2026-02-17T06:28:21.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj325 testdata.507902 2026-02-17T06:28:21.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj326 testdata.507902 2026-02-17T06:28:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj327 testdata.507902 2026-02-17T06:28:21.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj328 testdata.507902 2026-02-17T06:28:21.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj329 testdata.507902 2026-02-17T06:28:21.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj330 testdata.507902 2026-02-17T06:28:21.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj331 testdata.507902 2026-02-17T06:28:21.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj332 testdata.507902 2026-02-17T06:28:21.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj333 testdata.507902 2026-02-17T06:28:21.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj334 testdata.507902 2026-02-17T06:28:21.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj335 testdata.507902 2026-02-17T06:28:21.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj336 testdata.507902 2026-02-17T06:28:21.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj337 testdata.507902 2026-02-17T06:28:21.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj338 testdata.507902 2026-02-17T06:28:21.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj339 testdata.507902 2026-02-17T06:28:21.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj340 testdata.507902 2026-02-17T06:28:21.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj341 testdata.507902 2026-02-17T06:28:21.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj342 testdata.507902 2026-02-17T06:28:21.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj343 testdata.507902 2026-02-17T06:28:21.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj344 testdata.507902 2026-02-17T06:28:21.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj345 testdata.507902 2026-02-17T06:28:21.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj346 testdata.507902 2026-02-17T06:28:21.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj347 testdata.507902 2026-02-17T06:28:21.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj348 testdata.507902 2026-02-17T06:28:21.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj349 testdata.507902 2026-02-17T06:28:21.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj350 testdata.507902 2026-02-17T06:28:21.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj351 testdata.507902 2026-02-17T06:28:21.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj352 testdata.507902 2026-02-17T06:28:21.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj353 testdata.507902 2026-02-17T06:28:21.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj354 testdata.507902 2026-02-17T06:28:21.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj355 testdata.507902 2026-02-17T06:28:21.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj356 testdata.507902 2026-02-17T06:28:21.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj357 testdata.507902 2026-02-17T06:28:21.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj358 testdata.507902 2026-02-17T06:28:21.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj359 testdata.507902 2026-02-17T06:28:21.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj360 testdata.507902 2026-02-17T06:28:21.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:21.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj361 testdata.507902 2026-02-17T06:28:22.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj362 testdata.507902 2026-02-17T06:28:22.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj363 testdata.507902 2026-02-17T06:28:22.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj364 testdata.507902 2026-02-17T06:28:22.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj365 testdata.507902 2026-02-17T06:28:22.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj366 testdata.507902 2026-02-17T06:28:22.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj367 testdata.507902 2026-02-17T06:28:22.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj368 testdata.507902 2026-02-17T06:28:22.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj369 testdata.507902 2026-02-17T06:28:22.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj370 testdata.507902 2026-02-17T06:28:22.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj371 testdata.507902 2026-02-17T06:28:22.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj372 testdata.507902 2026-02-17T06:28:22.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj373 testdata.507902 2026-02-17T06:28:22.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj374 testdata.507902 2026-02-17T06:28:22.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj375 testdata.507902 2026-02-17T06:28:22.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj376 testdata.507902 2026-02-17T06:28:22.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj377 testdata.507902 2026-02-17T06:28:22.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj378 testdata.507902 2026-02-17T06:28:22.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj379 testdata.507902 2026-02-17T06:28:22.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj380 testdata.507902 2026-02-17T06:28:22.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj381 testdata.507902 2026-02-17T06:28:22.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj382 testdata.507902 2026-02-17T06:28:22.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj383 testdata.507902 2026-02-17T06:28:22.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj384 testdata.507902 2026-02-17T06:28:22.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj385 testdata.507902 2026-02-17T06:28:22.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj386 testdata.507902 2026-02-17T06:28:22.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj387 testdata.507902 2026-02-17T06:28:22.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj388 testdata.507902 2026-02-17T06:28:22.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj389 testdata.507902 2026-02-17T06:28:22.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj390 testdata.507902 2026-02-17T06:28:22.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj391 testdata.507902 2026-02-17T06:28:22.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj392 testdata.507902 2026-02-17T06:28:22.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj393 testdata.507902 2026-02-17T06:28:22.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj394 testdata.507902 2026-02-17T06:28:22.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj395 testdata.507902 2026-02-17T06:28:22.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj396 testdata.507902 2026-02-17T06:28:22.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj397 testdata.507902 2026-02-17T06:28:22.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj398 testdata.507902 2026-02-17T06:28:22.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj399 testdata.507902 2026-02-17T06:28:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj400 testdata.507902 2026-02-17T06:28:22.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj401 testdata.507902 2026-02-17T06:28:22.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:22.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj402 testdata.507902 2026-02-17T06:28:23.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj403 testdata.507902 2026-02-17T06:28:23.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj404 testdata.507902 2026-02-17T06:28:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj405 testdata.507902 2026-02-17T06:28:23.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj406 testdata.507902 2026-02-17T06:28:23.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj407 testdata.507902 2026-02-17T06:28:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj408 testdata.507902 2026-02-17T06:28:23.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj409 testdata.507902 2026-02-17T06:28:23.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj410 testdata.507902 2026-02-17T06:28:23.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj411 testdata.507902 2026-02-17T06:28:23.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj412 testdata.507902 2026-02-17T06:28:23.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj413 testdata.507902 2026-02-17T06:28:23.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj414 testdata.507902 2026-02-17T06:28:23.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj415 testdata.507902 2026-02-17T06:28:23.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj416 testdata.507902 2026-02-17T06:28:23.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj417 testdata.507902 2026-02-17T06:28:23.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj418 testdata.507902 2026-02-17T06:28:23.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj419 testdata.507902 2026-02-17T06:28:23.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj420 testdata.507902 2026-02-17T06:28:23.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj421 testdata.507902 2026-02-17T06:28:23.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj422 testdata.507902 2026-02-17T06:28:23.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj423 testdata.507902 2026-02-17T06:28:23.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj424 testdata.507902 2026-02-17T06:28:23.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj425 testdata.507902 2026-02-17T06:28:23.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj426 testdata.507902 2026-02-17T06:28:23.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj427 testdata.507902 2026-02-17T06:28:23.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj428 testdata.507902 2026-02-17T06:28:23.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj429 testdata.507902 2026-02-17T06:28:23.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj430 testdata.507902 2026-02-17T06:28:23.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj431 testdata.507902 2026-02-17T06:28:23.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj432 testdata.507902 2026-02-17T06:28:23.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj433 testdata.507902 2026-02-17T06:28:23.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj434 testdata.507902 2026-02-17T06:28:23.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj435 testdata.507902 2026-02-17T06:28:23.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj436 testdata.507902 2026-02-17T06:28:23.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj437 testdata.507902 2026-02-17T06:28:23.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj438 testdata.507902 2026-02-17T06:28:23.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj439 testdata.507902 2026-02-17T06:28:23.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj440 testdata.507902 2026-02-17T06:28:23.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj441 testdata.507902 2026-02-17T06:28:23.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj442 testdata.507902 2026-02-17T06:28:23.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj443 testdata.507902 2026-02-17T06:28:23.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:23.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj444 testdata.507902 2026-02-17T06:28:24.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj445 testdata.507902 2026-02-17T06:28:24.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj446 testdata.507902 2026-02-17T06:28:24.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj447 testdata.507902 2026-02-17T06:28:24.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj448 testdata.507902 2026-02-17T06:28:24.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj449 testdata.507902 2026-02-17T06:28:24.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj450 testdata.507902 2026-02-17T06:28:24.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj451 testdata.507902 2026-02-17T06:28:24.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj452 testdata.507902 2026-02-17T06:28:24.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj453 testdata.507902 2026-02-17T06:28:24.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj454 testdata.507902 2026-02-17T06:28:24.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj455 testdata.507902 2026-02-17T06:28:24.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj456 testdata.507902 2026-02-17T06:28:24.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj457 testdata.507902 2026-02-17T06:28:24.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj458 testdata.507902 2026-02-17T06:28:24.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj459 testdata.507902 2026-02-17T06:28:24.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj460 testdata.507902 2026-02-17T06:28:24.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj461 testdata.507902 2026-02-17T06:28:24.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj462 testdata.507902 2026-02-17T06:28:24.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj463 testdata.507902 2026-02-17T06:28:24.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj464 testdata.507902 2026-02-17T06:28:24.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj465 testdata.507902 2026-02-17T06:28:24.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj466 testdata.507902 2026-02-17T06:28:24.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj467 testdata.507902 2026-02-17T06:28:24.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj468 testdata.507902 2026-02-17T06:28:24.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj469 testdata.507902 2026-02-17T06:28:24.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj470 testdata.507902 2026-02-17T06:28:24.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj471 testdata.507902 2026-02-17T06:28:24.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj472 testdata.507902 2026-02-17T06:28:24.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj473 testdata.507902 2026-02-17T06:28:24.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj474 testdata.507902 2026-02-17T06:28:24.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj475 testdata.507902 2026-02-17T06:28:24.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj476 testdata.507902 2026-02-17T06:28:24.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj477 testdata.507902 2026-02-17T06:28:24.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj478 testdata.507902 2026-02-17T06:28:24.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj479 testdata.507902 2026-02-17T06:28:24.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj480 testdata.507902 2026-02-17T06:28:24.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj481 testdata.507902 2026-02-17T06:28:24.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj482 testdata.507902 2026-02-17T06:28:24.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj483 testdata.507902 2026-02-17T06:28:24.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj484 testdata.507902 2026-02-17T06:28:24.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj485 testdata.507902 2026-02-17T06:28:24.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj486 testdata.507902 2026-02-17T06:28:24.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:24.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj487 testdata.507902 2026-02-17T06:28:25.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj488 testdata.507902 2026-02-17T06:28:25.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj489 testdata.507902 2026-02-17T06:28:25.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj490 testdata.507902 2026-02-17T06:28:25.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj491 testdata.507902 2026-02-17T06:28:25.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj492 testdata.507902 2026-02-17T06:28:25.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj493 testdata.507902 2026-02-17T06:28:25.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj494 testdata.507902 2026-02-17T06:28:25.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj495 testdata.507902 2026-02-17T06:28:25.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj496 testdata.507902 2026-02-17T06:28:25.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj497 testdata.507902 2026-02-17T06:28:25.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj498 testdata.507902 2026-02-17T06:28:25.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj499 testdata.507902 2026-02-17T06:28:25.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj500 testdata.507902 2026-02-17T06:28:25.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj501 testdata.507902 2026-02-17T06:28:25.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj502 testdata.507902 2026-02-17T06:28:25.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj503 testdata.507902 2026-02-17T06:28:25.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj504 testdata.507902 2026-02-17T06:28:25.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj505 testdata.507902 2026-02-17T06:28:25.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj506 testdata.507902 2026-02-17T06:28:25.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj507 testdata.507902 2026-02-17T06:28:25.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj508 testdata.507902 2026-02-17T06:28:25.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj509 testdata.507902 2026-02-17T06:28:25.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj510 testdata.507902 2026-02-17T06:28:25.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj511 testdata.507902 2026-02-17T06:28:25.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj512 testdata.507902 2026-02-17T06:28:25.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj513 testdata.507902 2026-02-17T06:28:25.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj514 testdata.507902 2026-02-17T06:28:25.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj515 testdata.507902 2026-02-17T06:28:25.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj516 testdata.507902 2026-02-17T06:28:25.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj517 testdata.507902 2026-02-17T06:28:25.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj518 testdata.507902 2026-02-17T06:28:25.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj519 testdata.507902 2026-02-17T06:28:25.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj520 testdata.507902 2026-02-17T06:28:25.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj521 testdata.507902 2026-02-17T06:28:25.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj522 testdata.507902 2026-02-17T06:28:25.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj523 testdata.507902 2026-02-17T06:28:25.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj524 testdata.507902 2026-02-17T06:28:25.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj525 testdata.507902 2026-02-17T06:28:25.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj526 testdata.507902 2026-02-17T06:28:25.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj527 testdata.507902 2026-02-17T06:28:25.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj528 testdata.507902 2026-02-17T06:28:25.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:25.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj529 testdata.507902 2026-02-17T06:28:26.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj530 testdata.507902 2026-02-17T06:28:26.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj531 testdata.507902 2026-02-17T06:28:26.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj532 testdata.507902 2026-02-17T06:28:26.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj533 testdata.507902 2026-02-17T06:28:26.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj534 testdata.507902 2026-02-17T06:28:26.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj535 testdata.507902 2026-02-17T06:28:26.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj536 testdata.507902 2026-02-17T06:28:26.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj537 testdata.507902 2026-02-17T06:28:26.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj538 testdata.507902 2026-02-17T06:28:26.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj539 testdata.507902 2026-02-17T06:28:26.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj540 testdata.507902 2026-02-17T06:28:26.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj541 testdata.507902 2026-02-17T06:28:26.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj542 testdata.507902 2026-02-17T06:28:26.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj543 testdata.507902 2026-02-17T06:28:26.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj544 testdata.507902 2026-02-17T06:28:26.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj545 testdata.507902 2026-02-17T06:28:26.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj546 testdata.507902 2026-02-17T06:28:26.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj547 testdata.507902 2026-02-17T06:28:26.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj548 testdata.507902 2026-02-17T06:28:26.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj549 testdata.507902 2026-02-17T06:28:26.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj550 testdata.507902 2026-02-17T06:28:26.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj551 testdata.507902 2026-02-17T06:28:26.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj552 testdata.507902 2026-02-17T06:28:26.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj553 testdata.507902 2026-02-17T06:28:26.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj554 testdata.507902 2026-02-17T06:28:26.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj555 testdata.507902 2026-02-17T06:28:26.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj556 testdata.507902 2026-02-17T06:28:26.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj557 testdata.507902 2026-02-17T06:28:26.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj558 testdata.507902 2026-02-17T06:28:26.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj559 testdata.507902 2026-02-17T06:28:26.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj560 testdata.507902 2026-02-17T06:28:26.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj561 testdata.507902 2026-02-17T06:28:26.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj562 testdata.507902 2026-02-17T06:28:26.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj563 testdata.507902 2026-02-17T06:28:26.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj564 testdata.507902 2026-02-17T06:28:26.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj565 testdata.507902 2026-02-17T06:28:26.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj566 testdata.507902 2026-02-17T06:28:26.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj567 testdata.507902 2026-02-17T06:28:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj568 testdata.507902 2026-02-17T06:28:26.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj569 testdata.507902 2026-02-17T06:28:26.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj570 testdata.507902 2026-02-17T06:28:26.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:26.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj571 testdata.507902 2026-02-17T06:28:27.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj572 testdata.507902 2026-02-17T06:28:27.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj573 testdata.507902 2026-02-17T06:28:27.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj574 testdata.507902 2026-02-17T06:28:27.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj575 testdata.507902 2026-02-17T06:28:27.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj576 testdata.507902 2026-02-17T06:28:27.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj577 testdata.507902 2026-02-17T06:28:27.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj578 testdata.507902 2026-02-17T06:28:27.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj579 testdata.507902 2026-02-17T06:28:27.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj580 testdata.507902 2026-02-17T06:28:27.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj581 testdata.507902 2026-02-17T06:28:27.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj582 testdata.507902 2026-02-17T06:28:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj583 testdata.507902 2026-02-17T06:28:27.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj584 testdata.507902 2026-02-17T06:28:27.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj585 testdata.507902 2026-02-17T06:28:27.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj586 testdata.507902 2026-02-17T06:28:27.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj587 testdata.507902 2026-02-17T06:28:27.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj588 testdata.507902 2026-02-17T06:28:27.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj589 testdata.507902 2026-02-17T06:28:27.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj590 testdata.507902 2026-02-17T06:28:27.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj591 testdata.507902 2026-02-17T06:28:27.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj592 testdata.507902 2026-02-17T06:28:27.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj593 testdata.507902 2026-02-17T06:28:27.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj594 testdata.507902 2026-02-17T06:28:27.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj595 testdata.507902 2026-02-17T06:28:27.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj596 testdata.507902 2026-02-17T06:28:27.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj597 testdata.507902 2026-02-17T06:28:27.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj598 testdata.507902 2026-02-17T06:28:27.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj599 testdata.507902 2026-02-17T06:28:27.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj600 testdata.507902 2026-02-17T06:28:27.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj601 testdata.507902 2026-02-17T06:28:27.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj602 testdata.507902 2026-02-17T06:28:27.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj603 testdata.507902 2026-02-17T06:28:27.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj604 testdata.507902 2026-02-17T06:28:27.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj605 testdata.507902 2026-02-17T06:28:27.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj606 testdata.507902 2026-02-17T06:28:27.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj607 testdata.507902 2026-02-17T06:28:27.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj608 testdata.507902 2026-02-17T06:28:27.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj609 testdata.507902 2026-02-17T06:28:27.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj610 testdata.507902 2026-02-17T06:28:27.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj611 testdata.507902 2026-02-17T06:28:27.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:27.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj612 testdata.507902 2026-02-17T06:28:28.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj613 testdata.507902 2026-02-17T06:28:28.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj614 testdata.507902 2026-02-17T06:28:28.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj615 testdata.507902 2026-02-17T06:28:28.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj616 testdata.507902 2026-02-17T06:28:28.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj617 testdata.507902 2026-02-17T06:28:28.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj618 testdata.507902 2026-02-17T06:28:28.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj619 testdata.507902 2026-02-17T06:28:28.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj620 testdata.507902 2026-02-17T06:28:28.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj621 testdata.507902 2026-02-17T06:28:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj622 testdata.507902 2026-02-17T06:28:28.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj623 testdata.507902 2026-02-17T06:28:28.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj624 testdata.507902 2026-02-17T06:28:28.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj625 testdata.507902 2026-02-17T06:28:28.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj626 testdata.507902 2026-02-17T06:28:28.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj627 testdata.507902 2026-02-17T06:28:28.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj628 testdata.507902 2026-02-17T06:28:28.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj629 testdata.507902 2026-02-17T06:28:28.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj630 testdata.507902 2026-02-17T06:28:28.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj631 testdata.507902 2026-02-17T06:28:28.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj632 testdata.507902 2026-02-17T06:28:28.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj633 testdata.507902 2026-02-17T06:28:28.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj634 testdata.507902 2026-02-17T06:28:28.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj635 testdata.507902 2026-02-17T06:28:28.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj636 testdata.507902 2026-02-17T06:28:28.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj637 testdata.507902 2026-02-17T06:28:28.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj638 testdata.507902 2026-02-17T06:28:28.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj639 testdata.507902 2026-02-17T06:28:28.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj640 testdata.507902 2026-02-17T06:28:28.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj641 testdata.507902 2026-02-17T06:28:28.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj642 testdata.507902 2026-02-17T06:28:28.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj643 testdata.507902 2026-02-17T06:28:28.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj644 testdata.507902 2026-02-17T06:28:28.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj645 testdata.507902 2026-02-17T06:28:28.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj646 testdata.507902 2026-02-17T06:28:28.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj647 testdata.507902 2026-02-17T06:28:28.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj648 testdata.507902 2026-02-17T06:28:28.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj649 testdata.507902 2026-02-17T06:28:28.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj650 testdata.507902 2026-02-17T06:28:28.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj651 testdata.507902 2026-02-17T06:28:28.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj652 testdata.507902 2026-02-17T06:28:28.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj653 testdata.507902 2026-02-17T06:28:28.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:28.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj654 testdata.507902 2026-02-17T06:28:29.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj655 testdata.507902 2026-02-17T06:28:29.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj656 testdata.507902 2026-02-17T06:28:29.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj657 testdata.507902 2026-02-17T06:28:29.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj658 testdata.507902 2026-02-17T06:28:29.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj659 testdata.507902 2026-02-17T06:28:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj660 testdata.507902 2026-02-17T06:28:29.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj661 testdata.507902 2026-02-17T06:28:29.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj662 testdata.507902 2026-02-17T06:28:29.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj663 testdata.507902 2026-02-17T06:28:29.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj664 testdata.507902 2026-02-17T06:28:29.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj665 testdata.507902 2026-02-17T06:28:29.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj666 testdata.507902 2026-02-17T06:28:29.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj667 testdata.507902 2026-02-17T06:28:29.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj668 testdata.507902 2026-02-17T06:28:29.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj669 testdata.507902 2026-02-17T06:28:29.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj670 testdata.507902 2026-02-17T06:28:29.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj671 testdata.507902 2026-02-17T06:28:29.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj672 testdata.507902 2026-02-17T06:28:29.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj673 testdata.507902 2026-02-17T06:28:29.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj674 testdata.507902 2026-02-17T06:28:29.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj675 testdata.507902 2026-02-17T06:28:29.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj676 testdata.507902 2026-02-17T06:28:29.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj677 testdata.507902 2026-02-17T06:28:29.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj678 testdata.507902 2026-02-17T06:28:29.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj679 testdata.507902 2026-02-17T06:28:29.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj680 testdata.507902 2026-02-17T06:28:29.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj681 testdata.507902 2026-02-17T06:28:29.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj682 testdata.507902 2026-02-17T06:28:29.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj683 testdata.507902 2026-02-17T06:28:29.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj684 testdata.507902 2026-02-17T06:28:29.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj685 testdata.507902 2026-02-17T06:28:29.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj686 testdata.507902 2026-02-17T06:28:29.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj687 testdata.507902 2026-02-17T06:28:29.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj688 testdata.507902 2026-02-17T06:28:29.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj689 testdata.507902 2026-02-17T06:28:29.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj690 testdata.507902 2026-02-17T06:28:29.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj691 testdata.507902 2026-02-17T06:28:29.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj692 testdata.507902 2026-02-17T06:28:29.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj693 testdata.507902 2026-02-17T06:28:29.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj694 testdata.507902 2026-02-17T06:28:29.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj695 testdata.507902 2026-02-17T06:28:29.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj696 testdata.507902 2026-02-17T06:28:29.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj697 testdata.507902 2026-02-17T06:28:29.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj698 testdata.507902 2026-02-17T06:28:29.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj699 testdata.507902 2026-02-17T06:28:29.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:29.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj700 testdata.507902 2026-02-17T06:28:30.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj701 testdata.507902 2026-02-17T06:28:30.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj702 testdata.507902 2026-02-17T06:28:30.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj703 testdata.507902 2026-02-17T06:28:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj704 testdata.507902 2026-02-17T06:28:30.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj705 testdata.507902 2026-02-17T06:28:30.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj706 testdata.507902 2026-02-17T06:28:30.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj707 testdata.507902 2026-02-17T06:28:30.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj708 testdata.507902 2026-02-17T06:28:30.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj709 testdata.507902 2026-02-17T06:28:30.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj710 testdata.507902 2026-02-17T06:28:30.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj711 testdata.507902 2026-02-17T06:28:30.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj712 testdata.507902 2026-02-17T06:28:30.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj713 testdata.507902 2026-02-17T06:28:30.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj714 testdata.507902 2026-02-17T06:28:30.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj715 testdata.507902 2026-02-17T06:28:30.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj716 testdata.507902 2026-02-17T06:28:30.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj717 testdata.507902 2026-02-17T06:28:30.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj718 testdata.507902 2026-02-17T06:28:30.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj719 testdata.507902 2026-02-17T06:28:30.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj720 testdata.507902 2026-02-17T06:28:30.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj721 testdata.507902 2026-02-17T06:28:30.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj722 testdata.507902 2026-02-17T06:28:30.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj723 testdata.507902 2026-02-17T06:28:30.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj724 testdata.507902 2026-02-17T06:28:30.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj725 testdata.507902 2026-02-17T06:28:30.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj726 testdata.507902 2026-02-17T06:28:30.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj727 testdata.507902 2026-02-17T06:28:30.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj728 testdata.507902 2026-02-17T06:28:30.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj729 testdata.507902 2026-02-17T06:28:30.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj730 testdata.507902 2026-02-17T06:28:30.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj731 testdata.507902 2026-02-17T06:28:30.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj732 testdata.507902 2026-02-17T06:28:30.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj733 testdata.507902 2026-02-17T06:28:30.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj734 testdata.507902 2026-02-17T06:28:30.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj735 testdata.507902 2026-02-17T06:28:30.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj736 testdata.507902 2026-02-17T06:28:30.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj737 testdata.507902 2026-02-17T06:28:30.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj738 testdata.507902 2026-02-17T06:28:30.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj739 testdata.507902 2026-02-17T06:28:30.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj740 testdata.507902 2026-02-17T06:28:30.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj741 testdata.507902 2026-02-17T06:28:30.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:30.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj742 testdata.507902 2026-02-17T06:28:31.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj743 testdata.507902 2026-02-17T06:28:31.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj744 testdata.507902 2026-02-17T06:28:31.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj745 testdata.507902 2026-02-17T06:28:31.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj746 testdata.507902 2026-02-17T06:28:31.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj747 testdata.507902 2026-02-17T06:28:31.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj748 testdata.507902 2026-02-17T06:28:31.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj749 testdata.507902 2026-02-17T06:28:31.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj750 testdata.507902 2026-02-17T06:28:31.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj751 testdata.507902 2026-02-17T06:28:31.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj752 testdata.507902 2026-02-17T06:28:31.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj753 testdata.507902 2026-02-17T06:28:31.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj754 testdata.507902 2026-02-17T06:28:31.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj755 testdata.507902 2026-02-17T06:28:31.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj756 testdata.507902 2026-02-17T06:28:31.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj757 testdata.507902 2026-02-17T06:28:31.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj758 testdata.507902 2026-02-17T06:28:31.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj759 testdata.507902 2026-02-17T06:28:31.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj760 testdata.507902 2026-02-17T06:28:31.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj761 testdata.507902 2026-02-17T06:28:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj762 testdata.507902 2026-02-17T06:28:31.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj763 testdata.507902 2026-02-17T06:28:31.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj764 testdata.507902 2026-02-17T06:28:31.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj765 testdata.507902 2026-02-17T06:28:31.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj766 testdata.507902 2026-02-17T06:28:31.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj767 testdata.507902 2026-02-17T06:28:31.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj768 testdata.507902 2026-02-17T06:28:31.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj769 testdata.507902 2026-02-17T06:28:31.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj770 testdata.507902 2026-02-17T06:28:31.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj771 testdata.507902 2026-02-17T06:28:31.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj772 testdata.507902 2026-02-17T06:28:31.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj773 testdata.507902 2026-02-17T06:28:31.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj774 testdata.507902 2026-02-17T06:28:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj775 testdata.507902 2026-02-17T06:28:31.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj776 testdata.507902 2026-02-17T06:28:31.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj777 testdata.507902 2026-02-17T06:28:31.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj778 testdata.507902 2026-02-17T06:28:31.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj779 testdata.507902 2026-02-17T06:28:31.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj780 testdata.507902 2026-02-17T06:28:31.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj781 testdata.507902 2026-02-17T06:28:31.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj782 testdata.507902 2026-02-17T06:28:31.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:31.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj783 testdata.507902 2026-02-17T06:28:32.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj784 testdata.507902 2026-02-17T06:28:32.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj785 testdata.507902 2026-02-17T06:28:32.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj786 testdata.507902 2026-02-17T06:28:32.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj787 testdata.507902 2026-02-17T06:28:32.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj788 testdata.507902 2026-02-17T06:28:32.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj789 testdata.507902 2026-02-17T06:28:32.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj790 testdata.507902 2026-02-17T06:28:32.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj791 testdata.507902 2026-02-17T06:28:32.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj792 testdata.507902 2026-02-17T06:28:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj793 testdata.507902 2026-02-17T06:28:32.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj794 testdata.507902 2026-02-17T06:28:32.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj795 testdata.507902 2026-02-17T06:28:32.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj796 testdata.507902 2026-02-17T06:28:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj797 testdata.507902 2026-02-17T06:28:32.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj798 testdata.507902 2026-02-17T06:28:32.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj799 testdata.507902 2026-02-17T06:28:32.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj800 testdata.507902 2026-02-17T06:28:32.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj801 testdata.507902 2026-02-17T06:28:32.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj802 testdata.507902 2026-02-17T06:28:32.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj803 testdata.507902 2026-02-17T06:28:32.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj804 testdata.507902 2026-02-17T06:28:32.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj805 testdata.507902 2026-02-17T06:28:32.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj806 testdata.507902 2026-02-17T06:28:32.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj807 testdata.507902 2026-02-17T06:28:32.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj808 testdata.507902 2026-02-17T06:28:32.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj809 testdata.507902 2026-02-17T06:28:32.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj810 testdata.507902 2026-02-17T06:28:32.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj811 testdata.507902 2026-02-17T06:28:32.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj812 testdata.507902 2026-02-17T06:28:32.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj813 testdata.507902 2026-02-17T06:28:32.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj814 testdata.507902 2026-02-17T06:28:32.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj815 testdata.507902 2026-02-17T06:28:32.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj816 testdata.507902 2026-02-17T06:28:32.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj817 testdata.507902 2026-02-17T06:28:32.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj818 testdata.507902 2026-02-17T06:28:32.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj819 testdata.507902 2026-02-17T06:28:32.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj820 testdata.507902 2026-02-17T06:28:32.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj821 testdata.507902 2026-02-17T06:28:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj822 testdata.507902 2026-02-17T06:28:32.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj823 testdata.507902 2026-02-17T06:28:32.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj824 testdata.507902 2026-02-17T06:28:32.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:32.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj825 testdata.507902 2026-02-17T06:28:33.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj826 testdata.507902 2026-02-17T06:28:33.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj827 testdata.507902 2026-02-17T06:28:33.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj828 testdata.507902 2026-02-17T06:28:33.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj829 testdata.507902 2026-02-17T06:28:33.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj830 testdata.507902 2026-02-17T06:28:33.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj831 testdata.507902 2026-02-17T06:28:33.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj832 testdata.507902 2026-02-17T06:28:33.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj833 testdata.507902 2026-02-17T06:28:33.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj834 testdata.507902 2026-02-17T06:28:33.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj835 testdata.507902 2026-02-17T06:28:33.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj836 testdata.507902 2026-02-17T06:28:33.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj837 testdata.507902 2026-02-17T06:28:33.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj838 testdata.507902 2026-02-17T06:28:33.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj839 testdata.507902 2026-02-17T06:28:33.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj840 testdata.507902 2026-02-17T06:28:33.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj841 testdata.507902 2026-02-17T06:28:33.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj842 testdata.507902 2026-02-17T06:28:33.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj843 testdata.507902 2026-02-17T06:28:33.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj844 testdata.507902 2026-02-17T06:28:33.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj845 testdata.507902 2026-02-17T06:28:33.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj846 testdata.507902 2026-02-17T06:28:33.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj847 testdata.507902 2026-02-17T06:28:33.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj848 testdata.507902 2026-02-17T06:28:33.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj849 testdata.507902 2026-02-17T06:28:33.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj850 testdata.507902 2026-02-17T06:28:33.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj851 testdata.507902 2026-02-17T06:28:33.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj852 testdata.507902 2026-02-17T06:28:33.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj853 testdata.507902 2026-02-17T06:28:33.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj854 testdata.507902 2026-02-17T06:28:33.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj855 testdata.507902 2026-02-17T06:28:33.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj856 testdata.507902 2026-02-17T06:28:33.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj857 testdata.507902 2026-02-17T06:28:33.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj858 testdata.507902 2026-02-17T06:28:33.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj859 testdata.507902 2026-02-17T06:28:33.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj860 testdata.507902 2026-02-17T06:28:33.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj861 testdata.507902 2026-02-17T06:28:33.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj862 testdata.507902 2026-02-17T06:28:33.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj863 testdata.507902 2026-02-17T06:28:33.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj864 testdata.507902 2026-02-17T06:28:33.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj865 testdata.507902 2026-02-17T06:28:33.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj866 testdata.507902 2026-02-17T06:28:33.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:33.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj867 testdata.507902 2026-02-17T06:28:34.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj868 testdata.507902 2026-02-17T06:28:34.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj869 testdata.507902 2026-02-17T06:28:34.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj870 testdata.507902 2026-02-17T06:28:34.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj871 testdata.507902 2026-02-17T06:28:34.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj872 testdata.507902 2026-02-17T06:28:34.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj873 testdata.507902 2026-02-17T06:28:34.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj874 testdata.507902 2026-02-17T06:28:34.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj875 testdata.507902 2026-02-17T06:28:34.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj876 testdata.507902 2026-02-17T06:28:34.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj877 testdata.507902 2026-02-17T06:28:34.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj878 testdata.507902 2026-02-17T06:28:34.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj879 testdata.507902 2026-02-17T06:28:34.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj880 testdata.507902 2026-02-17T06:28:34.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj881 testdata.507902 2026-02-17T06:28:34.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj882 testdata.507902 2026-02-17T06:28:34.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj883 testdata.507902 2026-02-17T06:28:34.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj884 testdata.507902 2026-02-17T06:28:34.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj885 testdata.507902 2026-02-17T06:28:34.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj886 testdata.507902 2026-02-17T06:28:34.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj887 testdata.507902 2026-02-17T06:28:34.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj888 testdata.507902 2026-02-17T06:28:34.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj889 testdata.507902 2026-02-17T06:28:34.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj890 testdata.507902 2026-02-17T06:28:34.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj891 testdata.507902 2026-02-17T06:28:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj892 testdata.507902 2026-02-17T06:28:34.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj893 testdata.507902 2026-02-17T06:28:34.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj894 testdata.507902 2026-02-17T06:28:34.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj895 testdata.507902 2026-02-17T06:28:34.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj896 testdata.507902 2026-02-17T06:28:34.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj897 testdata.507902 2026-02-17T06:28:34.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj898 testdata.507902 2026-02-17T06:28:34.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj899 testdata.507902 2026-02-17T06:28:34.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj900 testdata.507902 2026-02-17T06:28:34.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj901 testdata.507902 2026-02-17T06:28:34.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj902 testdata.507902 2026-02-17T06:28:34.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj903 testdata.507902 2026-02-17T06:28:34.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj904 testdata.507902 2026-02-17T06:28:34.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj905 testdata.507902 2026-02-17T06:28:34.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj906 testdata.507902 2026-02-17T06:28:34.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj907 testdata.507902 2026-02-17T06:28:34.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj908 testdata.507902 2026-02-17T06:28:34.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:34.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj909 testdata.507902 2026-02-17T06:28:35.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj910 testdata.507902 2026-02-17T06:28:35.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj911 testdata.507902 2026-02-17T06:28:35.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj912 testdata.507902 2026-02-17T06:28:35.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj913 testdata.507902 2026-02-17T06:28:35.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj914 testdata.507902 2026-02-17T06:28:35.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj915 testdata.507902 2026-02-17T06:28:35.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj916 testdata.507902 2026-02-17T06:28:35.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj917 testdata.507902 2026-02-17T06:28:35.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj918 testdata.507902 2026-02-17T06:28:35.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj919 testdata.507902 2026-02-17T06:28:35.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj920 testdata.507902 2026-02-17T06:28:35.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj921 testdata.507902 2026-02-17T06:28:35.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj922 testdata.507902 2026-02-17T06:28:35.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj923 testdata.507902 2026-02-17T06:28:35.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj924 testdata.507902 2026-02-17T06:28:35.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj925 testdata.507902 2026-02-17T06:28:35.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj926 testdata.507902 2026-02-17T06:28:35.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj927 testdata.507902 2026-02-17T06:28:35.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj928 testdata.507902 2026-02-17T06:28:35.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj929 testdata.507902 2026-02-17T06:28:35.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj930 testdata.507902 2026-02-17T06:28:35.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj931 testdata.507902 2026-02-17T06:28:35.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj932 testdata.507902 2026-02-17T06:28:35.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj933 testdata.507902 2026-02-17T06:28:35.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj934 testdata.507902 2026-02-17T06:28:35.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj935 testdata.507902 2026-02-17T06:28:35.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj936 testdata.507902 2026-02-17T06:28:35.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj937 testdata.507902 2026-02-17T06:28:35.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj938 testdata.507902 2026-02-17T06:28:35.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj939 testdata.507902 2026-02-17T06:28:35.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj940 testdata.507902 2026-02-17T06:28:35.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj941 testdata.507902 2026-02-17T06:28:35.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj942 testdata.507902 2026-02-17T06:28:35.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj943 testdata.507902 2026-02-17T06:28:35.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj944 testdata.507902 2026-02-17T06:28:35.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj945 testdata.507902 2026-02-17T06:28:35.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj946 testdata.507902 2026-02-17T06:28:35.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj947 testdata.507902 2026-02-17T06:28:35.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj948 testdata.507902 2026-02-17T06:28:35.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj949 testdata.507902 2026-02-17T06:28:35.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj950 testdata.507902 2026-02-17T06:28:35.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:35.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj951 testdata.507902 2026-02-17T06:28:36.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj952 testdata.507902 2026-02-17T06:28:36.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj953 testdata.507902 2026-02-17T06:28:36.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj954 testdata.507902 2026-02-17T06:28:36.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj955 testdata.507902 2026-02-17T06:28:36.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj956 testdata.507902 2026-02-17T06:28:36.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj957 testdata.507902 2026-02-17T06:28:36.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj958 testdata.507902 2026-02-17T06:28:36.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj959 testdata.507902 2026-02-17T06:28:36.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj960 testdata.507902 2026-02-17T06:28:36.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj961 testdata.507902 2026-02-17T06:28:36.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj962 testdata.507902 2026-02-17T06:28:36.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj963 testdata.507902 2026-02-17T06:28:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj964 testdata.507902 2026-02-17T06:28:36.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj965 testdata.507902 2026-02-17T06:28:36.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj966 testdata.507902 2026-02-17T06:28:36.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj967 testdata.507902 2026-02-17T06:28:36.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj968 testdata.507902 2026-02-17T06:28:36.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj969 testdata.507902 2026-02-17T06:28:36.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj970 testdata.507902 2026-02-17T06:28:36.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj971 testdata.507902 2026-02-17T06:28:36.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj972 testdata.507902 2026-02-17T06:28:36.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj973 testdata.507902 2026-02-17T06:28:36.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj974 testdata.507902 2026-02-17T06:28:36.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj975 testdata.507902 2026-02-17T06:28:36.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj976 testdata.507902 2026-02-17T06:28:36.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj977 testdata.507902 2026-02-17T06:28:36.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj978 testdata.507902 2026-02-17T06:28:36.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj979 testdata.507902 2026-02-17T06:28:36.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj980 testdata.507902 2026-02-17T06:28:36.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj981 testdata.507902 2026-02-17T06:28:36.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj982 testdata.507902 2026-02-17T06:28:36.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj983 testdata.507902 2026-02-17T06:28:36.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj984 testdata.507902 2026-02-17T06:28:36.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj985 testdata.507902 2026-02-17T06:28:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj986 testdata.507902 2026-02-17T06:28:36.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj987 testdata.507902 2026-02-17T06:28:36.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj988 testdata.507902 2026-02-17T06:28:36.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj989 testdata.507902 2026-02-17T06:28:36.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj990 testdata.507902 2026-02-17T06:28:36.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj991 testdata.507902 2026-02-17T06:28:36.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj992 testdata.507902 2026-02-17T06:28:36.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:36.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj993 testdata.507902 2026-02-17T06:28:37.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:37.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj994 testdata.507902 2026-02-17T06:28:37.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:37.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj995 testdata.507902 2026-02-17T06:28:37.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:37.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj996 testdata.507902 2026-02-17T06:28:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:37.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj997 testdata.507902 2026-02-17T06:28:37.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:37.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj998 testdata.507902 2026-02-17T06:28:37.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:37.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj999 testdata.507902 2026-02-17T06:28:37.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:28:37.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1000 testdata.507902 2026-02-17T06:28:37.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:339: _scrub_abort: rm -f testdata.507902 2026-02-17T06:28:37.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: get_primary test obj1 2026-02-17T06:28:37.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:28:37.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:28:37.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:28:37.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:28:37.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: local primary=1 2026-02-17T06:28:37.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:342: _scrub_abort: local pgid=1.0 2026-02-17T06:28:37.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: ceph tell 1.0 schedule-deep-scrub 2026-02-17T06:28:37.386 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:28:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:28:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:28:37.387 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-03T06:26:57.383367+0000" 2026-02-17T06:28:37.387 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:28:37.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: set -o pipefail 2026-02-17T06:28:37.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:348: _scrub_abort: found=no 2026-02-17T06:28:37.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: seq 0 200 2026-02-17T06:28:37.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:37.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T06:28:37.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:37.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:37.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:37.546 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:37.546 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:37.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:37.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:37.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:37.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836525 2026-02-17T06:28:37.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836525 2026-02-17T06:28:37.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525' 2026-02-17T06:28:37.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:37.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:37.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705701 2026-02-17T06:28:37.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705701 2026-02-17T06:28:37.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705701' 2026-02-17T06:28:37.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:37.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:37.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542174 2026-02-17T06:28:37.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542174 2026-02-17T06:28:37.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705701 2-60129542174' 2026-02-17T06:28:37.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:37.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836525 2026-02-17T06:28:37.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:37.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:37.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836525 2026-02-17T06:28:37.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:37.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836525 2026-02-17T06:28:37.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836525' 2026-02-17T06:28:37.767 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836525 2026-02-17T06:28:37.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:37.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836523 -lt 21474836525 2026-02-17T06:28:37.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:38.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:38.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:39.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836525 2026-02-17T06:28:39.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:39.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705701 2026-02-17T06:28:39.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:39.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:39.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705701 2026-02-17T06:28:39.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:39.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705701 2026-02-17T06:28:39.084 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705701 2026-02-17T06:28:39.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-17T06:28:39.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:39.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705701 2026-02-17T06:28:39.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:39.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542174 2026-02-17T06:28:39.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:39.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:39.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542174 2026-02-17T06:28:39.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:39.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542174 2026-02-17T06:28:39.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542174' 2026-02-17T06:28:39.249 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542174 2026-02-17T06:28:39.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:39.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542174 -lt 60129542174 2026-02-17T06:28:39.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:39.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T06:28:39.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:39.543 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:39.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:39.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T06:28:39.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:39.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:39.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:39.702 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:39.702 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:39.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:39.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:39.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:39.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836528 2026-02-17T06:28:39.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836528 2026-02-17T06:28:39.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528' 2026-02-17T06:28:39.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:39.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:39.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705704 2026-02-17T06:28:39.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705704 2026-02-17T06:28:39.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705704' 2026-02-17T06:28:39.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:39.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:39.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542177 2026-02-17T06:28:39.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542177 2026-02-17T06:28:39.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705704 2-60129542177' 2026-02-17T06:28:39.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:39.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836528 2026-02-17T06:28:39.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:39.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:39.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836528 2026-02-17T06:28:39.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:39.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836528 2026-02-17T06:28:39.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836528' 2026-02-17T06:28:39.917 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836528 2026-02-17T06:28:39.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:40.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836528 2026-02-17T06:28:40.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:41.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:41.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836529 -lt 21474836528 2026-02-17T06:28:41.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:41.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705704 2026-02-17T06:28:41.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:41.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:41.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705704 2026-02-17T06:28:41.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:41.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705704 2026-02-17T06:28:41.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705704' 2026-02-17T06:28:41.222 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705704 2026-02-17T06:28:41.223 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:41.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705705 -lt 38654705704 2026-02-17T06:28:41.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:41.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542177 2026-02-17T06:28:41.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:41.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:41.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542177 2026-02-17T06:28:41.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:41.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542177 2026-02-17T06:28:41.384 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542177 2026-02-17T06:28:41.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542177' 2026-02-17T06:28:41.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:41.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542177 -lt 60129542177 2026-02-17T06:28:41.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:41.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T06:28:41.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:41.689 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:41.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:354: _scrub_abort: found=yes 2026-02-17T06:28:41.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:356: _scrub_abort: break 2026-02-17T06:28:41.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:359: _scrub_abort: set +o pipefail 2026-02-17T06:28:41.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:361: _scrub_abort: test yes = no 2026-02-17T06:28:41.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:367: _scrub_abort: ceph osd set nodeep-scrub 2026-02-17T06:28:41.892 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:28:41.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:368: _scrub_abort: '[' deep-scrub = deep-scrub ']' 2026-02-17T06:28:41.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: ceph osd set noscrub 2026-02-17T06:28:42.097 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:28:42.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:374: _scrub_abort: set -o pipefail 2026-02-17T06:28:42.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: seq 0 200 2026-02-17T06:28:42.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:42.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:42.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:42.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:42.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:42.261 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:42.261 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:42.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:42.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:42.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:42.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836531 2026-02-17T06:28:42.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836531 2026-02-17T06:28:42.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531' 2026-02-17T06:28:42.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:42.441 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:42.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705708 2026-02-17T06:28:42.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705708 2026-02-17T06:28:42.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-38654705708' 2026-02-17T06:28:42.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:42.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:42.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542180 2026-02-17T06:28:42.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542180 2026-02-17T06:28:42.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-38654705708 2-60129542180' 2026-02-17T06:28:42.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:42.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836531 2026-02-17T06:28:42.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:42.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:42.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836531 2026-02-17T06:28:42.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:42.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836531 2026-02-17T06:28:42.582 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836531 2026-02-17T06:28:42.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836531' 2026-02-17T06:28:42.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:42.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836532 -lt 21474836531 2026-02-17T06:28:42.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:42.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705708 2026-02-17T06:28:42.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:42.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:42.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705708 2026-02-17T06:28:42.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:42.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705708 2026-02-17T06:28:42.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705708' 2026-02-17T06:28:42.732 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705708 2026-02-17T06:28:42.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:42.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705708 -lt 38654705708 2026-02-17T06:28:42.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:42.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542180 2026-02-17T06:28:42.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:42.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:42.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542180 2026-02-17T06:28:42.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:42.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542180 2026-02-17T06:28:42.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542180' 2026-02-17T06:28:42.894 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542180 2026-02-17T06:28:42.895 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:43.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542179 -lt 60129542180 2026-02-17T06:28:43.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:44.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:44.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:44.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542179 -lt 60129542180 2026-02-17T06:28:44.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:45.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:28:45.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:45.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542182 -lt 60129542180 2026-02-17T06:28:45.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:45.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:45.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:45.498 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:45.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:45.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:45.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:45.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:45.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:45.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:45.658 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:45.658 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:45.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:45.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:45.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:45.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836536 2026-02-17T06:28:45.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836536 2026-02-17T06:28:45.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536' 2026-02-17T06:28:45.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:45.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:45.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705712 2026-02-17T06:28:45.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705712 2026-02-17T06:28:45.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-38654705712' 2026-02-17T06:28:45.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:45.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542185 2026-02-17T06:28:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542185 2026-02-17T06:28:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836536 1-38654705712 2-60129542185' 2026-02-17T06:28:45.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:45.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836536 2026-02-17T06:28:45.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:45.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:45.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836536 2026-02-17T06:28:45.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:45.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836536 2026-02-17T06:28:45.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836536' 2026-02-17T06:28:45.874 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836536 2026-02-17T06:28:45.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:46.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836536 2026-02-17T06:28:46.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:47.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:47.031 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:47.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836537 -lt 21474836536 2026-02-17T06:28:47.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:47.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705712 2026-02-17T06:28:47.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:47.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:47.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705712 2026-02-17T06:28:47.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:47.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705712 2026-02-17T06:28:47.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705712' 2026-02-17T06:28:47.187 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705712 2026-02-17T06:28:47.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:47.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705713 -lt 38654705712 2026-02-17T06:28:47.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:47.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542185 2026-02-17T06:28:47.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:47.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:47.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542185 2026-02-17T06:28:47.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:47.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542185 2026-02-17T06:28:47.349 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542185 2026-02-17T06:28:47.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542185' 2026-02-17T06:28:47.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:47.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542185 -lt 60129542185 2026-02-17T06:28:47.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:47.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:47.503 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:47.640 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:47.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:47.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:47.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:47.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:47.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:47.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:47.796 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:47.796 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:47.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:47.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:47.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:47.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836539 2026-02-17T06:28:47.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836539 2026-02-17T06:28:47.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539' 2026-02-17T06:28:47.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:47.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:47.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705715 2026-02-17T06:28:47.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705715 2026-02-17T06:28:47.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539 1-38654705715' 2026-02-17T06:28:47.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:47.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:48.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542188 2026-02-17T06:28:48.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542188 2026-02-17T06:28:48.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539 1-38654705715 2-60129542188' 2026-02-17T06:28:48.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:48.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836539 2026-02-17T06:28:48.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:48.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:48.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836539 2026-02-17T06:28:48.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:48.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836539 2026-02-17T06:28:48.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836539' 2026-02-17T06:28:48.010 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836539 2026-02-17T06:28:48.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:48.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836537 -lt 21474836539 2026-02-17T06:28:48.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:49.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:49.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:49.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836540 -lt 21474836539 2026-02-17T06:28:49.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:49.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705715 2026-02-17T06:28:49.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:49.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:49.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705715 2026-02-17T06:28:49.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:49.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705715 2026-02-17T06:28:49.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705715' 2026-02-17T06:28:49.315 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705715 2026-02-17T06:28:49.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:49.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705716 -lt 38654705715 2026-02-17T06:28:49.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:49.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542188 2026-02-17T06:28:49.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:49.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:49.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542188 2026-02-17T06:28:49.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:49.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542188 2026-02-17T06:28:49.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542188' 2026-02-17T06:28:49.477 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542188 2026-02-17T06:28:49.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:49.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542188 -lt 60129542188 2026-02-17T06:28:49.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:49.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:49.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:49.779 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:49.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:49.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:49.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:49.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:49.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:49.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:49.935 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:49.935 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:49.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:49.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:49.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836542 2026-02-17T06:28:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836542 2026-02-17T06:28:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836542' 2026-02-17T06:28:50.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:50.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:50.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705719 2026-02-17T06:28:50.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705719 2026-02-17T06:28:50.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836542 1-38654705719' 2026-02-17T06:28:50.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:50.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:50.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542191 2026-02-17T06:28:50.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542191 2026-02-17T06:28:50.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836542 1-38654705719 2-60129542191' 2026-02-17T06:28:50.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:50.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836542 2026-02-17T06:28:50.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:50.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:50.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836542 2026-02-17T06:28:50.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:50.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836542 2026-02-17T06:28:50.151 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836542 2026-02-17T06:28:50.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836542' 2026-02-17T06:28:50.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836540 -lt 21474836542 2026-02-17T06:28:50.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:51.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:51.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:51.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836543 -lt 21474836542 2026-02-17T06:28:51.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:51.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705719 2026-02-17T06:28:51.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:51.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:51.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705719 2026-02-17T06:28:51.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:51.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705719 2026-02-17T06:28:51.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705719' 2026-02-17T06:28:51.457 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705719 2026-02-17T06:28:51.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:51.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705719 -lt 38654705719 2026-02-17T06:28:51.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:51.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542191 2026-02-17T06:28:51.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:51.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:51.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542191 2026-02-17T06:28:51.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:51.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542191 2026-02-17T06:28:51.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542191' 2026-02-17T06:28:51.614 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542191 2026-02-17T06:28:51.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:51.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542191 -lt 60129542191 2026-02-17T06:28:51.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:51.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:51.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:51.914 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:51.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:51.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:51.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:51.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:51.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:52.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:52.076 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:52.076 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:52.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:52.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:52.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:52.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836545 2026-02-17T06:28:52.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836545 2026-02-17T06:28:52.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545' 2026-02-17T06:28:52.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:52.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:52.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705722 2026-02-17T06:28:52.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705722 2026-02-17T06:28:52.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545 1-38654705722' 2026-02-17T06:28:52.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:52.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:52.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542194 2026-02-17T06:28:52.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542194 2026-02-17T06:28:52.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545 1-38654705722 2-60129542194' 2026-02-17T06:28:52.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:52.290 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836545 2026-02-17T06:28:52.290 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:52.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:52.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836545 2026-02-17T06:28:52.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:52.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836545 2026-02-17T06:28:52.293 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836545 2026-02-17T06:28:52.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836545' 2026-02-17T06:28:52.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:52.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836543 -lt 21474836545 2026-02-17T06:28:52.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:53.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:53.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:53.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836546 -lt 21474836545 2026-02-17T06:28:53.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:53.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705722 2026-02-17T06:28:53.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:53.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:53.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705722 2026-02-17T06:28:53.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:53.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705722 2026-02-17T06:28:53.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705722' 2026-02-17T06:28:53.611 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705722 2026-02-17T06:28:53.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:53.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705722 -lt 38654705722 2026-02-17T06:28:53.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:53.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542194 2026-02-17T06:28:53.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:53.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:53.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542194 2026-02-17T06:28:53.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:53.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542194 2026-02-17T06:28:53.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542194' 2026-02-17T06:28:53.774 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542194 2026-02-17T06:28:53.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542194 -lt 60129542194 2026-02-17T06:28:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:53.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:54.069 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:54.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:54.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:54.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:54.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:54.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:54.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:54.232 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:54.232 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:54.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:54.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:54.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:54.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836548 2026-02-17T06:28:54.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836548 2026-02-17T06:28:54.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548' 2026-02-17T06:28:54.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:54.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:54.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705725 2026-02-17T06:28:54.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705725 2026-02-17T06:28:54.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548 1-38654705725' 2026-02-17T06:28:54.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:54.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:54.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542197 2026-02-17T06:28:54.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542197 2026-02-17T06:28:54.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548 1-38654705725 2-60129542197' 2026-02-17T06:28:54.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:54.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836548 2026-02-17T06:28:54.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:54.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:54.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836548 2026-02-17T06:28:54.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:54.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836548 2026-02-17T06:28:54.448 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836548 2026-02-17T06:28:54.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836548' 2026-02-17T06:28:54.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836549 -lt 21474836548 2026-02-17T06:28:54.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:54.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705725 2026-02-17T06:28:54.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:54.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:54.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705725 2026-02-17T06:28:54.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705725 2026-02-17T06:28:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705725' 2026-02-17T06:28:54.604 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705725 2026-02-17T06:28:54.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:54.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705725 -lt 38654705725 2026-02-17T06:28:54.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:54.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542197 2026-02-17T06:28:54.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:54.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:54.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542197 2026-02-17T06:28:54.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:54.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542197 2026-02-17T06:28:54.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542197' 2026-02-17T06:28:54.766 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542197 2026-02-17T06:28:54.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:54.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542197 -lt 60129542197 2026-02-17T06:28:54.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:54.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:54.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:55.069 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:55.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:55.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:55.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:55.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:55.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:55.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:55.230 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:55.230 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:55.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:55.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:55.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:55.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836550 2026-02-17T06:28:55.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836550 2026-02-17T06:28:55.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836550' 2026-02-17T06:28:55.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:55.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:55.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705727 2026-02-17T06:28:55.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705727 2026-02-17T06:28:55.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836550 1-38654705727' 2026-02-17T06:28:55.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:55.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:55.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542199 2026-02-17T06:28:55.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542199 2026-02-17T06:28:55.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836550 1-38654705727 2-60129542199' 2026-02-17T06:28:55.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:55.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836550 2026-02-17T06:28:55.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:55.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:55.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836550 2026-02-17T06:28:55.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:55.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836550 2026-02-17T06:28:55.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836550' 2026-02-17T06:28:55.449 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836550 2026-02-17T06:28:55.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:55.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836549 -lt 21474836550 2026-02-17T06:28:55.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:56.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:56.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:56.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836552 -lt 21474836550 2026-02-17T06:28:56.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:56.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705727 2026-02-17T06:28:56.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:56.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:56.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705727 2026-02-17T06:28:56.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:56.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705727 2026-02-17T06:28:56.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705727' 2026-02-17T06:28:56.756 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705727 2026-02-17T06:28:56.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:56.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705728 -lt 38654705727 2026-02-17T06:28:56.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:56.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542199 2026-02-17T06:28:56.915 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:56.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:56.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542199 2026-02-17T06:28:56.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:56.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542199 2026-02-17T06:28:56.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542199' 2026-02-17T06:28:56.918 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542199 2026-02-17T06:28:56.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:57.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542200 -lt 60129542199 2026-02-17T06:28:57.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:57.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:57.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:57.217 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:57.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:57.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:57.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:57.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:57.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:57.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:57.371 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:57.371 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:57.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:57.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:57.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:57.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836553 2026-02-17T06:28:57.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836553 2026-02-17T06:28:57.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836553' 2026-02-17T06:28:57.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:57.441 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:57.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705730 2026-02-17T06:28:57.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705730 2026-02-17T06:28:57.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836553 1-38654705730' 2026-02-17T06:28:57.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:57.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:57.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542202 2026-02-17T06:28:57.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542202 2026-02-17T06:28:57.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836553 1-38654705730 2-60129542202' 2026-02-17T06:28:57.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:57.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836553 2026-02-17T06:28:57.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:57.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:57.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836553 2026-02-17T06:28:57.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:57.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836553 2026-02-17T06:28:57.583 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836553 2026-02-17T06:28:57.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836553' 2026-02-17T06:28:57.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:57.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836552 -lt 21474836553 2026-02-17T06:28:57.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:28:58.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:28:58.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:58.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836555 -lt 21474836553 2026-02-17T06:28:58.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:58.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705730 2026-02-17T06:28:58.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:58.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:28:58.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705730 2026-02-17T06:28:58.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:58.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705730 2026-02-17T06:28:58.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705730' 2026-02-17T06:28:58.890 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705730 2026-02-17T06:28:58.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:28:59.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705731 -lt 38654705730 2026-02-17T06:28:59.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:59.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542202 2026-02-17T06:28:59.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:59.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:28:59.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542202 2026-02-17T06:28:59.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:59.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542202 2026-02-17T06:28:59.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542202' 2026-02-17T06:28:59.042 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542202 2026-02-17T06:28:59.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:28:59.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542203 -lt 60129542202 2026-02-17T06:28:59.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:28:59.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:28:59.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:28:59.336 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:28:59.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:28:59.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:28:59.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:28:59.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:28:59.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:28:59.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:28:59.496 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:28:59.496 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:28:59.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:28:59.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:59.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:28:59.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836557 2026-02-17T06:28:59.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836557 2026-02-17T06:28:59.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836557' 2026-02-17T06:28:59.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:59.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:28:59.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705733 2026-02-17T06:28:59.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705733 2026-02-17T06:28:59.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836557 1-38654705733' 2026-02-17T06:28:59.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:28:59.637 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:28:59.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542206 2026-02-17T06:28:59.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542206 2026-02-17T06:28:59.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836557 1-38654705733 2-60129542206' 2026-02-17T06:28:59.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:28:59.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836557 2026-02-17T06:28:59.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:28:59.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:28:59.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836557 2026-02-17T06:28:59.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:28:59.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836557 2026-02-17T06:28:59.710 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836557 2026-02-17T06:28:59.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836557' 2026-02-17T06:28:59.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:28:59.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836555 -lt 21474836557 2026-02-17T06:28:59.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:00.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:00.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:01.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836558 -lt 21474836557 2026-02-17T06:29:01.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:01.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705733 2026-02-17T06:29:01.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:01.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:01.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705733 2026-02-17T06:29:01.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:01.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705733 2026-02-17T06:29:01.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705733' 2026-02-17T06:29:01.024 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705733 2026-02-17T06:29:01.024 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:01.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705734 -lt 38654705733 2026-02-17T06:29:01.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:01.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542206 2026-02-17T06:29:01.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:01.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:01.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542206 2026-02-17T06:29:01.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:01.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542206 2026-02-17T06:29:01.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542206' 2026-02-17T06:29:01.179 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542206 2026-02-17T06:29:01.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:01.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542206 -lt 60129542206 2026-02-17T06:29:01.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:01.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:01.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:01.477 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:01.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:01.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:01.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:01.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:01.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:01.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:01.633 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:01.633 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:01.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:01.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:01.634 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:01.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836560 2026-02-17T06:29:01.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836560 2026-02-17T06:29:01.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836560' 2026-02-17T06:29:01.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:01.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:01.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705736 2026-02-17T06:29:01.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705736 2026-02-17T06:29:01.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836560 1-38654705736' 2026-02-17T06:29:01.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:01.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:01.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542209 2026-02-17T06:29:01.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542209 2026-02-17T06:29:01.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836560 1-38654705736 2-60129542209' 2026-02-17T06:29:01.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:01.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836560 2026-02-17T06:29:01.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:01.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:01.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836560 2026-02-17T06:29:01.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:01.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836560 2026-02-17T06:29:01.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836560' 2026-02-17T06:29:01.854 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836560 2026-02-17T06:29:01.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:02.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836558 -lt 21474836560 2026-02-17T06:29:02.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:03.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:03.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:03.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836561 -lt 21474836560 2026-02-17T06:29:03.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:03.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705736 2026-02-17T06:29:03.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:03.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:03.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705736 2026-02-17T06:29:03.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:03.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705736 2026-02-17T06:29:03.165 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705736 2026-02-17T06:29:03.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705736' 2026-02-17T06:29:03.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:03.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705737 -lt 38654705736 2026-02-17T06:29:03.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:03.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542209 2026-02-17T06:29:03.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:03.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:03.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542209 2026-02-17T06:29:03.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:03.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542209 2026-02-17T06:29:03.315 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542209 2026-02-17T06:29:03.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542209' 2026-02-17T06:29:03.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:03.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542209 -lt 60129542209 2026-02-17T06:29:03.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:03.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:03.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:03.605 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:03.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:03.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:03.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:03.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:03.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:03.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:03.772 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:03.772 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:03.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:03.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:03.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:03.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836563 2026-02-17T06:29:03.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836563 2026-02-17T06:29:03.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836563' 2026-02-17T06:29:03.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:03.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:03.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705739 2026-02-17T06:29:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705739 2026-02-17T06:29:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836563 1-38654705739' 2026-02-17T06:29:03.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:03.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:03.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542212 2026-02-17T06:29:03.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542212 2026-02-17T06:29:03.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836563 1-38654705739 2-60129542212' 2026-02-17T06:29:03.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:03.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836563 2026-02-17T06:29:03.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:03.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:03.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836563 2026-02-17T06:29:03.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:03.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836563 2026-02-17T06:29:03.987 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836563 2026-02-17T06:29:03.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836563' 2026-02-17T06:29:03.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:04.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836561 -lt 21474836563 2026-02-17T06:29:04.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:05.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:05.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:05.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836564 -lt 21474836563 2026-02-17T06:29:05.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:05.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705739 2026-02-17T06:29:05.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:05.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:05.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705739 2026-02-17T06:29:05.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:05.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705739 2026-02-17T06:29:05.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705739' 2026-02-17T06:29:05.295 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705739 2026-02-17T06:29:05.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:05.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705740 -lt 38654705739 2026-02-17T06:29:05.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:05.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542212 2026-02-17T06:29:05.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:05.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:05.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542212 2026-02-17T06:29:05.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:05.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542212 2026-02-17T06:29:05.459 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542212 2026-02-17T06:29:05.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542212' 2026-02-17T06:29:05.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:05.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542212 -lt 60129542212 2026-02-17T06:29:05.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:05.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:05.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:05.762 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:05.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:05.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:05.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:05.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:05.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:05.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:05.921 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:05.922 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:05.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:05.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:05.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:05.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836566 2026-02-17T06:29:05.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836566 2026-02-17T06:29:05.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836566' 2026-02-17T06:29:05.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:05.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:06.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705743 2026-02-17T06:29:06.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705743 2026-02-17T06:29:06.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836566 1-38654705743' 2026-02-17T06:29:06.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:06.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:06.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542215 2026-02-17T06:29:06.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542215 2026-02-17T06:29:06.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836566 1-38654705743 2-60129542215' 2026-02-17T06:29:06.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:06.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836566 2026-02-17T06:29:06.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:06.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:06.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836566 2026-02-17T06:29:06.140 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:06.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836566 2026-02-17T06:29:06.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836566' 2026-02-17T06:29:06.142 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836566 2026-02-17T06:29:06.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:06.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836564 -lt 21474836566 2026-02-17T06:29:06.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:07.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:07.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:07.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836567 -lt 21474836566 2026-02-17T06:29:07.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:07.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705743 2026-02-17T06:29:07.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:07.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:07.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705743 2026-02-17T06:29:07.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:07.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705743 2026-02-17T06:29:07.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705743' 2026-02-17T06:29:07.459 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705743 2026-02-17T06:29:07.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:07.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705743 -lt 38654705743 2026-02-17T06:29:07.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:07.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542215 2026-02-17T06:29:07.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:07.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:07.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542215 2026-02-17T06:29:07.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:07.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542215 2026-02-17T06:29:07.623 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542215 2026-02-17T06:29:07.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542215' 2026-02-17T06:29:07.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:07.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542215 -lt 60129542215 2026-02-17T06:29:07.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:07.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:07.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:07.914 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:07.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:07.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:07.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:07.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:07.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:08.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:08.069 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:08.069 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:08.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:08.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:08.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836569 2026-02-17T06:29:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836569 2026-02-17T06:29:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569' 2026-02-17T06:29:08.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:08.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705746 2026-02-17T06:29:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705746 2026-02-17T06:29:08.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569 1-38654705746' 2026-02-17T06:29:08.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:08.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:08.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542218 2026-02-17T06:29:08.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542218 2026-02-17T06:29:08.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569 1-38654705746 2-60129542218' 2026-02-17T06:29:08.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:08.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836569 2026-02-17T06:29:08.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:08.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:08.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836569 2026-02-17T06:29:08.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:08.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836569 2026-02-17T06:29:08.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836569' 2026-02-17T06:29:08.289 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836569 2026-02-17T06:29:08.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:08.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836567 -lt 21474836569 2026-02-17T06:29:08.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:09.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:09.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:09.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836570 -lt 21474836569 2026-02-17T06:29:09.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:09.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705746 2026-02-17T06:29:09.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:09.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:09.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705746 2026-02-17T06:29:09.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:09.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705746 2026-02-17T06:29:09.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705746' 2026-02-17T06:29:09.593 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705746 2026-02-17T06:29:09.593 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:09.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705746 -lt 38654705746 2026-02-17T06:29:09.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:09.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542218 2026-02-17T06:29:09.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:09.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542218 2026-02-17T06:29:09.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542218 2026-02-17T06:29:09.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542218' 2026-02-17T06:29:09.758 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542218 2026-02-17T06:29:09.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:09.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542218 -lt 60129542218 2026-02-17T06:29:09.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:09.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:09.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:10.052 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:10.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:10.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:10.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:10.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:10.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:10.207 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:10.207 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:10.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:10.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:10.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836572 2026-02-17T06:29:10.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836572 2026-02-17T06:29:10.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572' 2026-02-17T06:29:10.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:10.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:10.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705749 2026-02-17T06:29:10.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705749 2026-02-17T06:29:10.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572 1-38654705749' 2026-02-17T06:29:10.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:10.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:10.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542221 2026-02-17T06:29:10.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542221 2026-02-17T06:29:10.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572 1-38654705749 2-60129542221' 2026-02-17T06:29:10.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:10.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836572 2026-02-17T06:29:10.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:10.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:10.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836572 2026-02-17T06:29:10.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:10.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836572 2026-02-17T06:29:10.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836572' 2026-02-17T06:29:10.425 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836572 2026-02-17T06:29:10.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:10.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836570 -lt 21474836572 2026-02-17T06:29:10.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:11.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:11.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:11.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836573 -lt 21474836572 2026-02-17T06:29:11.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:11.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705749 2026-02-17T06:29:11.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:11.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:11.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705749 2026-02-17T06:29:11.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:11.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705749 2026-02-17T06:29:11.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705749' 2026-02-17T06:29:11.734 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705749 2026-02-17T06:29:11.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:11.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705749 -lt 38654705749 2026-02-17T06:29:11.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:11.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542221 2026-02-17T06:29:11.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:11.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:11.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542221 2026-02-17T06:29:11.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:11.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542221 2026-02-17T06:29:11.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542221' 2026-02-17T06:29:11.887 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542221 2026-02-17T06:29:11.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:12.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542221 -lt 60129542221 2026-02-17T06:29:12.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:12.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:12.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:12.186 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:12.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:12.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:12.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:12.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:12.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:12.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:12.341 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:12.341 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:12.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:12.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:12.342 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:12.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836575 2026-02-17T06:29:12.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836575 2026-02-17T06:29:12.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575' 2026-02-17T06:29:12.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:12.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:12.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705752 2026-02-17T06:29:12.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705752 2026-02-17T06:29:12.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575 1-38654705752' 2026-02-17T06:29:12.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:12.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:12.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542224 2026-02-17T06:29:12.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542224 2026-02-17T06:29:12.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575 1-38654705752 2-60129542224' 2026-02-17T06:29:12.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:12.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836575 2026-02-17T06:29:12.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:12.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:12.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836575 2026-02-17T06:29:12.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:12.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836575 2026-02-17T06:29:12.558 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836575 2026-02-17T06:29:12.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836575' 2026-02-17T06:29:12.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:12.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836576 -lt 21474836575 2026-02-17T06:29:12.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:12.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705752 2026-02-17T06:29:12.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:12.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:12.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705752 2026-02-17T06:29:12.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:12.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705752 2026-02-17T06:29:12.720 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705752 2026-02-17T06:29:12.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705752' 2026-02-17T06:29:12.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:12.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705752 -lt 38654705752 2026-02-17T06:29:12.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:12.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542224 2026-02-17T06:29:12.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:12.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:12.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542224 2026-02-17T06:29:12.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:12.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542224 2026-02-17T06:29:12.877 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542224 2026-02-17T06:29:12.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542224' 2026-02-17T06:29:12.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:13.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542223 -lt 60129542224 2026-02-17T06:29:13.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:14.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:14.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:14.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542223 -lt 60129542224 2026-02-17T06:29:14.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:15.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:29:15.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:15.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542226 -lt 60129542224 2026-02-17T06:29:15.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:15.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:15.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:15.469 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:15.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:15.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:15.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:15.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:15.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:15.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:15.633 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:15.634 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:15.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:15.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:15.634 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:15.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836580 2026-02-17T06:29:15.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836580 2026-02-17T06:29:15.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836580' 2026-02-17T06:29:15.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:15.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:15.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705756 2026-02-17T06:29:15.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705756 2026-02-17T06:29:15.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836580 1-38654705756' 2026-02-17T06:29:15.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:15.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:15.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542229 2026-02-17T06:29:15.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542229 2026-02-17T06:29:15.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836580 1-38654705756 2-60129542229' 2026-02-17T06:29:15.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:15.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836580 2026-02-17T06:29:15.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:15.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:15.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836580 2026-02-17T06:29:15.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:15.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836580 2026-02-17T06:29:15.849 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836580 2026-02-17T06:29:15.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836580' 2026-02-17T06:29:15.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:16.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836578 -lt 21474836580 2026-02-17T06:29:16.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:17.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:17.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:17.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836581 -lt 21474836580 2026-02-17T06:29:17.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:17.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705756 2026-02-17T06:29:17.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:17.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:17.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705756 2026-02-17T06:29:17.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:17.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705756 2026-02-17T06:29:17.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705756' 2026-02-17T06:29:17.158 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705756 2026-02-17T06:29:17.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:17.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705757 -lt 38654705756 2026-02-17T06:29:17.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:17.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542229 2026-02-17T06:29:17.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:17.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:17.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542229 2026-02-17T06:29:17.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:17.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542229 2026-02-17T06:29:17.315 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542229 2026-02-17T06:29:17.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542229' 2026-02-17T06:29:17.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:17.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542229 -lt 60129542229 2026-02-17T06:29:17.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:17.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:17.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:17.612 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:17.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:17.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:17.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:17.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:17.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:17.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:17.768 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:17.768 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:17.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:17.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:17.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:17.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836583 2026-02-17T06:29:17.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836583 2026-02-17T06:29:17.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836583' 2026-02-17T06:29:17.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:17.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705759 2026-02-17T06:29:17.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705759 2026-02-17T06:29:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836583 1-38654705759' 2026-02-17T06:29:17.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:17.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:17.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542232 2026-02-17T06:29:17.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542232 2026-02-17T06:29:17.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836583 1-38654705759 2-60129542232' 2026-02-17T06:29:17.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:17.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836583 2026-02-17T06:29:17.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:17.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:17.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836583 2026-02-17T06:29:17.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:17.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836583 2026-02-17T06:29:17.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836583' 2026-02-17T06:29:17.986 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836583 2026-02-17T06:29:17.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:18.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836581 -lt 21474836583 2026-02-17T06:29:18.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:19.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:19.150 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:19.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836584 -lt 21474836583 2026-02-17T06:29:19.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:19.299 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705759 2026-02-17T06:29:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:19.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:19.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705759 2026-02-17T06:29:19.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:19.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705759 2026-02-17T06:29:19.303 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705759 2026-02-17T06:29:19.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705759' 2026-02-17T06:29:19.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:19.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705760 -lt 38654705759 2026-02-17T06:29:19.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:19.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542232 2026-02-17T06:29:19.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:19.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:19.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542232 2026-02-17T06:29:19.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:19.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542232 2026-02-17T06:29:19.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542232' 2026-02-17T06:29:19.467 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542232 2026-02-17T06:29:19.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:19.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542232 -lt 60129542232 2026-02-17T06:29:19.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:19.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:19.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:19.752 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:19.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:19.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:19.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:19.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:19.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:19.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:19.912 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:19.912 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:19.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:19.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:19.913 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:19.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836586 2026-02-17T06:29:19.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836586 2026-02-17T06:29:19.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836586' 2026-02-17T06:29:19.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:19.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:20.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705763 2026-02-17T06:29:20.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705763 2026-02-17T06:29:20.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836586 1-38654705763' 2026-02-17T06:29:20.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:20.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:20.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542235 2026-02-17T06:29:20.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542235 2026-02-17T06:29:20.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836586 1-38654705763 2-60129542235' 2026-02-17T06:29:20.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:20.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836586 2026-02-17T06:29:20.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:20.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:20.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836586 2026-02-17T06:29:20.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:20.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836586 2026-02-17T06:29:20.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836586' 2026-02-17T06:29:20.131 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836586 2026-02-17T06:29:20.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:20.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836584 -lt 21474836586 2026-02-17T06:29:20.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:21.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:21.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:21.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836587 -lt 21474836586 2026-02-17T06:29:21.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:21.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705763 2026-02-17T06:29:21.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:21.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:21.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705763 2026-02-17T06:29:21.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:21.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705763 2026-02-17T06:29:21.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705763' 2026-02-17T06:29:21.447 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705763 2026-02-17T06:29:21.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:21.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705763 -lt 38654705763 2026-02-17T06:29:21.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:21.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542235 2026-02-17T06:29:21.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:21.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:21.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542235 2026-02-17T06:29:21.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:21.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542235 2026-02-17T06:29:21.609 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542235 2026-02-17T06:29:21.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542235' 2026-02-17T06:29:21.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:21.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542235 -lt 60129542235 2026-02-17T06:29:21.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:21.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:21.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:21.910 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:21.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:21.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:21.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:21.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:21.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:22.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:22.069 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:22.069 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:22.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:22.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:22.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:22.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836589 2026-02-17T06:29:22.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836589 2026-02-17T06:29:22.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836589' 2026-02-17T06:29:22.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:22.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:22.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705766 2026-02-17T06:29:22.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705766 2026-02-17T06:29:22.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836589 1-38654705766' 2026-02-17T06:29:22.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:22.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:22.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542238 2026-02-17T06:29:22.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542238 2026-02-17T06:29:22.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836589 1-38654705766 2-60129542238' 2026-02-17T06:29:22.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:22.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836589 2026-02-17T06:29:22.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:22.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:22.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836589 2026-02-17T06:29:22.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:22.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836589 2026-02-17T06:29:22.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836589' 2026-02-17T06:29:22.287 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836589 2026-02-17T06:29:22.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:22.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836587 -lt 21474836589 2026-02-17T06:29:22.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:23.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:23.446 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:23.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836590 -lt 21474836589 2026-02-17T06:29:23.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:23.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705766 2026-02-17T06:29:23.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:23.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:23.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705766 2026-02-17T06:29:23.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:23.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705766 2026-02-17T06:29:23.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705766' 2026-02-17T06:29:23.605 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705766 2026-02-17T06:29:23.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:23.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705766 -lt 38654705766 2026-02-17T06:29:23.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:23.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542238 2026-02-17T06:29:23.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:23.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:23.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542238 2026-02-17T06:29:23.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:23.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542238 2026-02-17T06:29:23.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542238' 2026-02-17T06:29:23.756 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542238 2026-02-17T06:29:23.757 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:23.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542238 -lt 60129542238 2026-02-17T06:29:23.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:23.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:23.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:24.057 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:24.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:24.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:24.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:24.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:24.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:24.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:24.213 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:24.213 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:24.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:24.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:24.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:24.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836592 2026-02-17T06:29:24.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836592 2026-02-17T06:29:24.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836592' 2026-02-17T06:29:24.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:24.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:24.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705769 2026-02-17T06:29:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705769 2026-02-17T06:29:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836592 1-38654705769' 2026-02-17T06:29:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:24.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:24.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542241 2026-02-17T06:29:24.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542241 2026-02-17T06:29:24.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836592 1-38654705769 2-60129542241' 2026-02-17T06:29:24.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:24.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836592 2026-02-17T06:29:24.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:24.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:24.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836592 2026-02-17T06:29:24.436 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:24.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836592 2026-02-17T06:29:24.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836592' 2026-02-17T06:29:24.438 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836592 2026-02-17T06:29:24.438 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:24.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836590 -lt 21474836592 2026-02-17T06:29:24.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:25.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:25.587 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:25.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836593 -lt 21474836592 2026-02-17T06:29:25.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:25.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705769 2026-02-17T06:29:25.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:25.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:25.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705769 2026-02-17T06:29:25.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:25.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705769 2026-02-17T06:29:25.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705769' 2026-02-17T06:29:25.744 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705769 2026-02-17T06:29:25.744 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:25.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705769 -lt 38654705769 2026-02-17T06:29:25.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:25.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542241 2026-02-17T06:29:25.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:25.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:25.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542241 2026-02-17T06:29:25.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:25.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542241 2026-02-17T06:29:25.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542241' 2026-02-17T06:29:25.905 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542241 2026-02-17T06:29:25.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:26.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542241 -lt 60129542241 2026-02-17T06:29:26.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:26.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:26.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:26.194 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:26.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:26.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:26.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:26.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:26.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:26.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:26.356 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:26.356 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:26.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:26.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:26.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:26.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836595 2026-02-17T06:29:26.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836595 2026-02-17T06:29:26.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595' 2026-02-17T06:29:26.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:26.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:26.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705772 2026-02-17T06:29:26.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705772 2026-02-17T06:29:26.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595 1-38654705772' 2026-02-17T06:29:26.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:26.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:26.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542244 2026-02-17T06:29:26.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542244 2026-02-17T06:29:26.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595 1-38654705772 2-60129542244' 2026-02-17T06:29:26.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:26.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836595 2026-02-17T06:29:26.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:26.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:26.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836595 2026-02-17T06:29:26.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:26.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836595 2026-02-17T06:29:26.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836595' 2026-02-17T06:29:26.575 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836595 2026-02-17T06:29:26.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:26.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836596 -lt 21474836595 2026-02-17T06:29:26.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:26.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705772 2026-02-17T06:29:26.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:26.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:26.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705772 2026-02-17T06:29:26.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:26.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705772 2026-02-17T06:29:26.736 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705772 2026-02-17T06:29:26.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705772' 2026-02-17T06:29:26.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705772 -lt 38654705772 2026-02-17T06:29:26.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:26.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542244 2026-02-17T06:29:26.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:26.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:26.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542244 2026-02-17T06:29:26.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:26.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542244 2026-02-17T06:29:26.893 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542244 2026-02-17T06:29:26.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542244' 2026-02-17T06:29:26.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:27.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542243 -lt 60129542244 2026-02-17T06:29:27.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:28.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:28.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:28.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542243 -lt 60129542244 2026-02-17T06:29:28.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:29.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:29:29.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:29.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542246 -lt 60129542244 2026-02-17T06:29:29.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:29.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:29.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:29.494 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:29.505 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:29.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:29.661 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:29.661 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:29.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:29.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:29.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:29.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836600 2026-02-17T06:29:29.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836600 2026-02-17T06:29:29.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836600' 2026-02-17T06:29:29.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:29.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:29.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705776 2026-02-17T06:29:29.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705776 2026-02-17T06:29:29.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836600 1-38654705776' 2026-02-17T06:29:29.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:29.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:29.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542249 2026-02-17T06:29:29.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542249 2026-02-17T06:29:29.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836600 1-38654705776 2-60129542249' 2026-02-17T06:29:29.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:29.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836600 2026-02-17T06:29:29.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:29.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:29.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836600 2026-02-17T06:29:29.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:29.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836600 2026-02-17T06:29:29.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836600' 2026-02-17T06:29:29.883 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836600 2026-02-17T06:29:29.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:30.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836598 -lt 21474836600 2026-02-17T06:29:30.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:31.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:31.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836601 -lt 21474836600 2026-02-17T06:29:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:31.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705776 2026-02-17T06:29:31.195 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:31.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:31.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705776 2026-02-17T06:29:31.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705776 2026-02-17T06:29:31.199 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705776 2026-02-17T06:29:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705776' 2026-02-17T06:29:31.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:31.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705777 -lt 38654705776 2026-02-17T06:29:31.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:31.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542249 2026-02-17T06:29:31.362 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:31.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:31.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542249 2026-02-17T06:29:31.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:31.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542249 2026-02-17T06:29:31.366 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542249 2026-02-17T06:29:31.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542249' 2026-02-17T06:29:31.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:31.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542249 -lt 60129542249 2026-02-17T06:29:31.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:31.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:31.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:31.667 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:31.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:31.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:31.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:31.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:31.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:31.824 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:31.824 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:31.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:31.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:31.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:31.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836603 2026-02-17T06:29:31.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836603 2026-02-17T06:29:31.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836603' 2026-02-17T06:29:31.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:31.895 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:31.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705779 2026-02-17T06:29:31.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705779 2026-02-17T06:29:31.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836603 1-38654705779' 2026-02-17T06:29:31.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:31.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:32.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542252 2026-02-17T06:29:32.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542252 2026-02-17T06:29:32.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836603 1-38654705779 2-60129542252' 2026-02-17T06:29:32.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:32.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836603 2026-02-17T06:29:32.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:32.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:32.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836603 2026-02-17T06:29:32.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836603 2026-02-17T06:29:32.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836603' 2026-02-17T06:29:32.042 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836603 2026-02-17T06:29:32.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:32.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836601 -lt 21474836603 2026-02-17T06:29:32.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:33.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:33.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:33.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836604 -lt 21474836603 2026-02-17T06:29:33.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:33.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705779 2026-02-17T06:29:33.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:33.356 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:33.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705779 2026-02-17T06:29:33.357 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:33.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705779 2026-02-17T06:29:33.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705779' 2026-02-17T06:29:33.358 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705779 2026-02-17T06:29:33.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:33.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705780 -lt 38654705779 2026-02-17T06:29:33.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:33.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542252 2026-02-17T06:29:33.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:33.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:33.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542252 2026-02-17T06:29:33.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:33.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542252 2026-02-17T06:29:33.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542252' 2026-02-17T06:29:33.523 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542252 2026-02-17T06:29:33.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:33.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542252 -lt 60129542252 2026-02-17T06:29:33.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:33.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:33.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:33.823 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:33.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:33.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:33.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:33.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:33.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:33.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:33.982 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:33.982 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:33.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:33.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:33.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:34.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836606 2026-02-17T06:29:34.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836606 2026-02-17T06:29:34.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836606' 2026-02-17T06:29:34.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:34.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:34.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705783 2026-02-17T06:29:34.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705783 2026-02-17T06:29:34.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836606 1-38654705783' 2026-02-17T06:29:34.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:34.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:34.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542255 2026-02-17T06:29:34.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542255 2026-02-17T06:29:34.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836606 1-38654705783 2-60129542255' 2026-02-17T06:29:34.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:34.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836606 2026-02-17T06:29:34.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:34.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:34.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836606 2026-02-17T06:29:34.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:34.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836606 2026-02-17T06:29:34.201 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836606 2026-02-17T06:29:34.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836606' 2026-02-17T06:29:34.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:34.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836604 -lt 21474836606 2026-02-17T06:29:34.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:35.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:35.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:35.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836607 -lt 21474836606 2026-02-17T06:29:35.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:35.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705783 2026-02-17T06:29:35.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:35.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:35.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705783 2026-02-17T06:29:35.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:35.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705783 2026-02-17T06:29:35.514 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705783 2026-02-17T06:29:35.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705783' 2026-02-17T06:29:35.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:35.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705783 -lt 38654705783 2026-02-17T06:29:35.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:35.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542255 2026-02-17T06:29:35.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:35.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:35.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542255 2026-02-17T06:29:35.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:35.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542255 2026-02-17T06:29:35.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542255' 2026-02-17T06:29:35.667 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542255 2026-02-17T06:29:35.667 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:35.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542255 -lt 60129542255 2026-02-17T06:29:35.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:35.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:35.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:35.964 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:35.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:35.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:35.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:35.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:35.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:36.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:36.132 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:36.132 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:36.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:36.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:36.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:36.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836609 2026-02-17T06:29:36.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836609 2026-02-17T06:29:36.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609' 2026-02-17T06:29:36.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:36.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:36.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705786 2026-02-17T06:29:36.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705786 2026-02-17T06:29:36.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609 1-38654705786' 2026-02-17T06:29:36.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:36.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:36.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542258 2026-02-17T06:29:36.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542258 2026-02-17T06:29:36.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609 1-38654705786 2-60129542258' 2026-02-17T06:29:36.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:36.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836609 2026-02-17T06:29:36.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:36.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:36.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836609 2026-02-17T06:29:36.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:36.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836609 2026-02-17T06:29:36.350 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836609 2026-02-17T06:29:36.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836609' 2026-02-17T06:29:36.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:36.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836607 -lt 21474836609 2026-02-17T06:29:36.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:37.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:37.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:37.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836610 -lt 21474836609 2026-02-17T06:29:37.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:37.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705786 2026-02-17T06:29:37.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:37.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:37.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705786 2026-02-17T06:29:37.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:37.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705786 2026-02-17T06:29:37.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705786' 2026-02-17T06:29:37.665 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705786 2026-02-17T06:29:37.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:37.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705786 -lt 38654705786 2026-02-17T06:29:37.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:37.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542258 2026-02-17T06:29:37.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:37.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:37.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542258 2026-02-17T06:29:37.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:37.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542258 2026-02-17T06:29:37.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542258' 2026-02-17T06:29:37.819 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542258 2026-02-17T06:29:37.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:37.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542258 -lt 60129542258 2026-02-17T06:29:37.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:37.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:37.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:38.117 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:38.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:38.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:38.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:38.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:38.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:38.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:38.274 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:38.274 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:38.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:38.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:38.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:38.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836612 2026-02-17T06:29:38.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836612 2026-02-17T06:29:38.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612' 2026-02-17T06:29:38.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:38.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:38.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705789 2026-02-17T06:29:38.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705789 2026-02-17T06:29:38.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612 1-38654705789' 2026-02-17T06:29:38.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:38.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:38.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542261 2026-02-17T06:29:38.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542261 2026-02-17T06:29:38.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612 1-38654705789 2-60129542261' 2026-02-17T06:29:38.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:38.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836612 2026-02-17T06:29:38.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:38.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:38.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836612 2026-02-17T06:29:38.488 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:38.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836612 2026-02-17T06:29:38.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836612' 2026-02-17T06:29:38.490 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836612 2026-02-17T06:29:38.490 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:38.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836613 -lt 21474836612 2026-02-17T06:29:38.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:38.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705789 2026-02-17T06:29:38.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:38.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:38.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705789 2026-02-17T06:29:38.642 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:38.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705789 2026-02-17T06:29:38.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705789' 2026-02-17T06:29:38.644 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705789 2026-02-17T06:29:38.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:38.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705789 -lt 38654705789 2026-02-17T06:29:38.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:38.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542261 2026-02-17T06:29:38.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:38.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:38.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542261 2026-02-17T06:29:38.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:38.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542261 2026-02-17T06:29:38.805 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542261 2026-02-17T06:29:38.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542261' 2026-02-17T06:29:38.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:38.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542261 -lt 60129542261 2026-02-17T06:29:38.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:38.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:38.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:39.098 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:39.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:39.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:39.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:39.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:39.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:39.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:39.260 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:39.260 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:39.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:39.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:39.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836614 2026-02-17T06:29:39.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836614 2026-02-17T06:29:39.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836614' 2026-02-17T06:29:39.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:39.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:39.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705791 2026-02-17T06:29:39.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705791 2026-02-17T06:29:39.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836614 1-38654705791' 2026-02-17T06:29:39.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:39.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:39.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542263 2026-02-17T06:29:39.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542263 2026-02-17T06:29:39.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836614 1-38654705791 2-60129542263' 2026-02-17T06:29:39.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:39.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836614 2026-02-17T06:29:39.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:39.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:39.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836614 2026-02-17T06:29:39.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:39.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836614 2026-02-17T06:29:39.478 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836614 2026-02-17T06:29:39.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836614' 2026-02-17T06:29:39.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:39.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836613 -lt 21474836614 2026-02-17T06:29:39.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:40.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:40.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:40.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836616 -lt 21474836614 2026-02-17T06:29:40.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:40.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705791 2026-02-17T06:29:40.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:40.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:40.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705791 2026-02-17T06:29:40.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:40.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705791 2026-02-17T06:29:40.794 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705791 2026-02-17T06:29:40.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705791' 2026-02-17T06:29:40.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:40.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705792 -lt 38654705791 2026-02-17T06:29:40.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:40.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542263 2026-02-17T06:29:40.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:40.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:40.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542263 2026-02-17T06:29:40.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:40.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542263 2026-02-17T06:29:40.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542263' 2026-02-17T06:29:40.954 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542263 2026-02-17T06:29:40.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:41.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542264 -lt 60129542263 2026-02-17T06:29:41.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:41.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:41.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:41.239 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:41.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:41.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:41.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:41.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:41.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:41.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:41.405 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:41.405 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:41.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:41.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:41.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:41.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836617 2026-02-17T06:29:41.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836617 2026-02-17T06:29:41.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836617' 2026-02-17T06:29:41.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:41.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:41.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705794 2026-02-17T06:29:41.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705794 2026-02-17T06:29:41.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836617 1-38654705794' 2026-02-17T06:29:41.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:41.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:41.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542267 2026-02-17T06:29:41.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542267 2026-02-17T06:29:41.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836617 1-38654705794 2-60129542267' 2026-02-17T06:29:41.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:41.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836617 2026-02-17T06:29:41.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:41.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:41.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836617 2026-02-17T06:29:41.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:41.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836617 2026-02-17T06:29:41.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836617' 2026-02-17T06:29:41.626 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836617 2026-02-17T06:29:41.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:41.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836616 -lt 21474836617 2026-02-17T06:29:41.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:42.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:42.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:42.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836619 -lt 21474836617 2026-02-17T06:29:42.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:42.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705794 2026-02-17T06:29:42.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:42.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:42.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705794 2026-02-17T06:29:42.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:42.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705794 2026-02-17T06:29:42.946 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705794 2026-02-17T06:29:42.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705794' 2026-02-17T06:29:42.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:43.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705795 -lt 38654705794 2026-02-17T06:29:43.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:43.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542267 2026-02-17T06:29:43.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:43.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:43.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542267 2026-02-17T06:29:43.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:43.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542267 2026-02-17T06:29:43.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542267' 2026-02-17T06:29:43.112 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542267 2026-02-17T06:29:43.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:43.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542267 -lt 60129542267 2026-02-17T06:29:43.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:43.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:43.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:43.415 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:43.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:43.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:43.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:43.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:43.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:43.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:43.576 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:43.576 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:43.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:43.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:43.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:43.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836621 2026-02-17T06:29:43.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836621 2026-02-17T06:29:43.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836621' 2026-02-17T06:29:43.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:43.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:43.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705797 2026-02-17T06:29:43.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705797 2026-02-17T06:29:43.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836621 1-38654705797' 2026-02-17T06:29:43.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:43.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:43.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542270 2026-02-17T06:29:43.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542270 2026-02-17T06:29:43.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836621 1-38654705797 2-60129542270' 2026-02-17T06:29:43.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:43.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836621 2026-02-17T06:29:43.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:43.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:43.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836621 2026-02-17T06:29:43.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:43.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836621 2026-02-17T06:29:43.793 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836621 2026-02-17T06:29:43.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836621' 2026-02-17T06:29:43.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:43.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836619 -lt 21474836621 2026-02-17T06:29:43.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:44.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:44.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:45.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836622 -lt 21474836621 2026-02-17T06:29:45.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:45.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705797 2026-02-17T06:29:45.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:45.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:45.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705797 2026-02-17T06:29:45.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:45.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705797 2026-02-17T06:29:45.107 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705797 2026-02-17T06:29:45.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705797' 2026-02-17T06:29:45.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:45.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705798 -lt 38654705797 2026-02-17T06:29:45.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:45.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542270 2026-02-17T06:29:45.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:45.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:45.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542270 2026-02-17T06:29:45.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:45.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542270 2026-02-17T06:29:45.271 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542270 2026-02-17T06:29:45.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542270' 2026-02-17T06:29:45.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:45.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542270 -lt 60129542270 2026-02-17T06:29:45.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:45.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:45.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:45.573 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:45.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:45.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:45.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:45.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:45.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:45.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:45.731 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:45.731 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:45.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:45.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:45.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:45.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836624 2026-02-17T06:29:45.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836624 2026-02-17T06:29:45.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836624' 2026-02-17T06:29:45.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:45.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:45.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705800 2026-02-17T06:29:45.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705800 2026-02-17T06:29:45.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836624 1-38654705800' 2026-02-17T06:29:45.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:45.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:45.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542273 2026-02-17T06:29:45.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542273 2026-02-17T06:29:45.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836624 1-38654705800 2-60129542273' 2026-02-17T06:29:45.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:45.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836624 2026-02-17T06:29:45.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:45.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:45.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836624 2026-02-17T06:29:45.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:45.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836624 2026-02-17T06:29:45.949 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836624 2026-02-17T06:29:45.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836624' 2026-02-17T06:29:45.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:46.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836622 -lt 21474836624 2026-02-17T06:29:46.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:47.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:47.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:47.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836625 -lt 21474836624 2026-02-17T06:29:47.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:47.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705800 2026-02-17T06:29:47.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:47.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:47.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705800 2026-02-17T06:29:47.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:47.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705800 2026-02-17T06:29:47.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705800' 2026-02-17T06:29:47.260 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705800 2026-02-17T06:29:47.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:47.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705801 -lt 38654705800 2026-02-17T06:29:47.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:47.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542273 2026-02-17T06:29:47.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:47.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:47.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542273 2026-02-17T06:29:47.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:47.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542273 2026-02-17T06:29:47.425 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542273 2026-02-17T06:29:47.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542273' 2026-02-17T06:29:47.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:47.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542273 -lt 60129542273 2026-02-17T06:29:47.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:47.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:47.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:47.725 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:47.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:47.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:47.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:47.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:47.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:47.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:47.880 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:47.880 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:47.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:47.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:47.880 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:47.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836627 2026-02-17T06:29:47.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836627 2026-02-17T06:29:47.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836627' 2026-02-17T06:29:47.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:47.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:48.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705804 2026-02-17T06:29:48.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705804 2026-02-17T06:29:48.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836627 1-38654705804' 2026-02-17T06:29:48.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:48.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:48.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542276 2026-02-17T06:29:48.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542276 2026-02-17T06:29:48.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836627 1-38654705804 2-60129542276' 2026-02-17T06:29:48.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:48.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836627 2026-02-17T06:29:48.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:48.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:48.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836627 2026-02-17T06:29:48.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:48.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836627 2026-02-17T06:29:48.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836627' 2026-02-17T06:29:48.098 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836627 2026-02-17T06:29:48.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:48.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836625 -lt 21474836627 2026-02-17T06:29:48.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:49.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:49.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:49.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836628 -lt 21474836627 2026-02-17T06:29:49.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:49.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705804 2026-02-17T06:29:49.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:49.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:49.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705804 2026-02-17T06:29:49.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705804 2026-02-17T06:29:49.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705804' 2026-02-17T06:29:49.416 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705804 2026-02-17T06:29:49.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:49.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705804 -lt 38654705804 2026-02-17T06:29:49.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:49.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542276 2026-02-17T06:29:49.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:49.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:49.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542276 2026-02-17T06:29:49.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:49.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542276 2026-02-17T06:29:49.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542276' 2026-02-17T06:29:49.579 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542276 2026-02-17T06:29:49.580 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:49.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542276 -lt 60129542276 2026-02-17T06:29:49.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:49.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:49.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:49.878 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:49.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:49.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:49.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:50.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:50.033 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:50.034 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:50.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:50.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:50.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:50.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836630 2026-02-17T06:29:50.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836630 2026-02-17T06:29:50.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836630' 2026-02-17T06:29:50.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:50.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:50.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705807 2026-02-17T06:29:50.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705807 2026-02-17T06:29:50.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836630 1-38654705807' 2026-02-17T06:29:50.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:50.174 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:50.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542279 2026-02-17T06:29:50.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542279 2026-02-17T06:29:50.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836630 1-38654705807 2-60129542279' 2026-02-17T06:29:50.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:50.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836630 2026-02-17T06:29:50.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:50.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:50.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836630 2026-02-17T06:29:50.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:50.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836630 2026-02-17T06:29:50.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836630' 2026-02-17T06:29:50.247 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836630 2026-02-17T06:29:50.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:50.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836628 -lt 21474836630 2026-02-17T06:29:50.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:51.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:51.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:51.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836631 -lt 21474836630 2026-02-17T06:29:51.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:51.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705807 2026-02-17T06:29:51.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:51.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:51.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705807 2026-02-17T06:29:51.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:51.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705807 2026-02-17T06:29:51.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705807' 2026-02-17T06:29:51.559 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705807 2026-02-17T06:29:51.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:51.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705807 -lt 38654705807 2026-02-17T06:29:51.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:51.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542279 2026-02-17T06:29:51.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:51.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:51.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542279 2026-02-17T06:29:51.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:51.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542279 2026-02-17T06:29:51.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542279' 2026-02-17T06:29:51.727 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542279 2026-02-17T06:29:51.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:51.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542279 -lt 60129542279 2026-02-17T06:29:51.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:51.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:51.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:52.025 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:52.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:52.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:52.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:52.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:52.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:52.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:52.184 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:52.184 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:52.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:52.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:52.185 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:52.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836633 2026-02-17T06:29:52.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836633 2026-02-17T06:29:52.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836633' 2026-02-17T06:29:52.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:52.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:52.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705810 2026-02-17T06:29:52.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705810 2026-02-17T06:29:52.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836633 1-38654705810' 2026-02-17T06:29:52.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:52.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:52.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542282 2026-02-17T06:29:52.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542282 2026-02-17T06:29:52.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836633 1-38654705810 2-60129542282' 2026-02-17T06:29:52.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:52.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836633 2026-02-17T06:29:52.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:52.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:52.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836633 2026-02-17T06:29:52.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:52.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836633 2026-02-17T06:29:52.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836633' 2026-02-17T06:29:52.402 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836633 2026-02-17T06:29:52.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:52.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836631 -lt 21474836633 2026-02-17T06:29:52.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:53.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:53.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:53.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836634 -lt 21474836633 2026-02-17T06:29:53.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:53.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705810 2026-02-17T06:29:53.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:53.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:53.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705810 2026-02-17T06:29:53.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:53.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705810 2026-02-17T06:29:53.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705810' 2026-02-17T06:29:53.711 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705810 2026-02-17T06:29:53.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:53.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705810 -lt 38654705810 2026-02-17T06:29:53.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:53.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542282 2026-02-17T06:29:53.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:53.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:53.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542282 2026-02-17T06:29:53.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:53.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542282 2026-02-17T06:29:53.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542282' 2026-02-17T06:29:53.874 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542282 2026-02-17T06:29:53.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:54.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542282 -lt 60129542282 2026-02-17T06:29:54.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:54.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:54.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:54.163 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:54.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:54.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:54.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:54.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:54.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:54.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:54.320 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:54.320 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:54.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:54.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:54.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:54.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836636 2026-02-17T06:29:54.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836636 2026-02-17T06:29:54.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836636' 2026-02-17T06:29:54.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:54.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:54.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705813 2026-02-17T06:29:54.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705813 2026-02-17T06:29:54.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836636 1-38654705813' 2026-02-17T06:29:54.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:54.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:54.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542285 2026-02-17T06:29:54.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542285 2026-02-17T06:29:54.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836636 1-38654705813 2-60129542285' 2026-02-17T06:29:54.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:54.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836636 2026-02-17T06:29:54.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:54.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:54.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836636 2026-02-17T06:29:54.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:54.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836636 2026-02-17T06:29:54.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836636' 2026-02-17T06:29:54.538 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836636 2026-02-17T06:29:54.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:54.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836637 -lt 21474836636 2026-02-17T06:29:54.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:54.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705813 2026-02-17T06:29:54.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:54.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:54.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705813 2026-02-17T06:29:54.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:54.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705813 2026-02-17T06:29:54.690 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705813 2026-02-17T06:29:54.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705813' 2026-02-17T06:29:54.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:54.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705813 -lt 38654705813 2026-02-17T06:29:54.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:54.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542285 2026-02-17T06:29:54.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:54.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:54.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542285 2026-02-17T06:29:54.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:54.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542285 2026-02-17T06:29:54.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542285' 2026-02-17T06:29:54.853 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542285 2026-02-17T06:29:54.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:55.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542285 -lt 60129542285 2026-02-17T06:29:55.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:55.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:55.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:55.143 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:55.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:55.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:55.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:55.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:55.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:55.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:55.305 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:55.305 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:55.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:55.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:55.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:55.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836638 2026-02-17T06:29:55.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836638 2026-02-17T06:29:55.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836638' 2026-02-17T06:29:55.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:55.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:55.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705815 2026-02-17T06:29:55.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705815 2026-02-17T06:29:55.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836638 1-38654705815' 2026-02-17T06:29:55.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:55.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:55.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542287 2026-02-17T06:29:55.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542287 2026-02-17T06:29:55.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836638 1-38654705815 2-60129542287' 2026-02-17T06:29:55.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:55.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836638 2026-02-17T06:29:55.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:55.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:55.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836638 2026-02-17T06:29:55.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:55.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836638 2026-02-17T06:29:55.525 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836638 2026-02-17T06:29:55.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836638' 2026-02-17T06:29:55.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:55.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836637 -lt 21474836638 2026-02-17T06:29:55.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:56.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:56.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:56.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836640 -lt 21474836638 2026-02-17T06:29:56.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:56.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705815 2026-02-17T06:29:56.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:56.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:56.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705815 2026-02-17T06:29:56.837 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:56.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705815 2026-02-17T06:29:56.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705815' 2026-02-17T06:29:56.838 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705815 2026-02-17T06:29:56.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:56.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705816 -lt 38654705815 2026-02-17T06:29:56.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:56.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542287 2026-02-17T06:29:56.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:57.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:57.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542287 2026-02-17T06:29:57.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542287 2026-02-17T06:29:57.002 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542287 2026-02-17T06:29:57.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542287' 2026-02-17T06:29:57.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:57.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542288 -lt 60129542287 2026-02-17T06:29:57.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:57.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:57.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:57.303 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:57.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:57.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:57.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:57.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:57.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:57.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:57.457 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:57.457 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:57.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:57.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:57.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:57.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836642 2026-02-17T06:29:57.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836642 2026-02-17T06:29:57.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836642' 2026-02-17T06:29:57.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:57.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:57.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705818 2026-02-17T06:29:57.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705818 2026-02-17T06:29:57.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836642 1-38654705818' 2026-02-17T06:29:57.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:57.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:57.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542291 2026-02-17T06:29:57.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542291 2026-02-17T06:29:57.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836642 1-38654705818 2-60129542291' 2026-02-17T06:29:57.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836642 2026-02-17T06:29:57.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:57.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:57.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836642 2026-02-17T06:29:57.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:57.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836642 2026-02-17T06:29:57.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836642' 2026-02-17T06:29:57.673 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836642 2026-02-17T06:29:57.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:57.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836640 -lt 21474836642 2026-02-17T06:29:57.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:29:58.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:29:58.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:58.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836643 -lt 21474836642 2026-02-17T06:29:58.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:58.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705818 2026-02-17T06:29:58.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:58.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:29:58.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705818 2026-02-17T06:29:58.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:58.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705818 2026-02-17T06:29:58.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705818' 2026-02-17T06:29:58.977 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705818 2026-02-17T06:29:58.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:29:59.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705819 -lt 38654705818 2026-02-17T06:29:59.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:59.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542291 2026-02-17T06:29:59.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:59.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:29:59.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542291 2026-02-17T06:29:59.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:59.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542291 2026-02-17T06:29:59.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542291' 2026-02-17T06:29:59.131 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542291 2026-02-17T06:29:59.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:29:59.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542291 -lt 60129542291 2026-02-17T06:29:59.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:29:59.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:29:59.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:29:59.429 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:29:59.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:29:59.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:29:59.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:29:59.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:29:59.439 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:29:59.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:29:59.588 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:29:59.589 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:29:59.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:29:59.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:59.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:29:59.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836645 2026-02-17T06:29:59.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836645 2026-02-17T06:29:59.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836645' 2026-02-17T06:29:59.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:59.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:29:59.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705821 2026-02-17T06:29:59.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705821 2026-02-17T06:29:59.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836645 1-38654705821' 2026-02-17T06:29:59.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:29:59.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:29:59.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542294 2026-02-17T06:29:59.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542294 2026-02-17T06:29:59.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836645 1-38654705821 2-60129542294' 2026-02-17T06:29:59.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:29:59.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836645 2026-02-17T06:29:59.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:29:59.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:29:59.803 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836645 2026-02-17T06:29:59.804 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:29:59.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836645 2026-02-17T06:29:59.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836645' 2026-02-17T06:29:59.806 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836645 2026-02-17T06:29:59.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:29:59.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836643 -lt 21474836645 2026-02-17T06:29:59.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:00.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:00.958 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:01.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836646 -lt 21474836645 2026-02-17T06:30:01.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:01.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705821 2026-02-17T06:30:01.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:01.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:01.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705821 2026-02-17T06:30:01.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:01.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705821 2026-02-17T06:30:01.111 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705821 2026-02-17T06:30:01.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705821' 2026-02-17T06:30:01.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:01.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705822 -lt 38654705821 2026-02-17T06:30:01.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:01.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542294 2026-02-17T06:30:01.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:01.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:01.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542294 2026-02-17T06:30:01.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:01.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542294 2026-02-17T06:30:01.269 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542294 2026-02-17T06:30:01.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542294' 2026-02-17T06:30:01.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:01.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542294 -lt 60129542294 2026-02-17T06:30:01.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:01.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:01.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:01.556 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:01.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:01.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:01.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:01.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:01.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:01.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:01.713 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:01.713 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:01.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:01.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:01.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:01.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836648 2026-02-17T06:30:01.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836648 2026-02-17T06:30:01.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836648' 2026-02-17T06:30:01.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:01.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:01.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705824 2026-02-17T06:30:01.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705824 2026-02-17T06:30:01.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836648 1-38654705824' 2026-02-17T06:30:01.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:01.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:01.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542297 2026-02-17T06:30:01.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542297 2026-02-17T06:30:01.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836648 1-38654705824 2-60129542297' 2026-02-17T06:30:01.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:01.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836648 2026-02-17T06:30:01.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:01.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:01.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836648 2026-02-17T06:30:01.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:01.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836648 2026-02-17T06:30:01.937 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836648 2026-02-17T06:30:01.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836648' 2026-02-17T06:30:01.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:02.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836646 -lt 21474836648 2026-02-17T06:30:02.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:03.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:03.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:03.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836649 -lt 21474836648 2026-02-17T06:30:03.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:03.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705824 2026-02-17T06:30:03.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:03.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:03.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705824 2026-02-17T06:30:03.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:03.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705824 2026-02-17T06:30:03.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705824' 2026-02-17T06:30:03.244 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705824 2026-02-17T06:30:03.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:03.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705825 -lt 38654705824 2026-02-17T06:30:03.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:03.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542297 2026-02-17T06:30:03.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:03.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:03.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542297 2026-02-17T06:30:03.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:03.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542297 2026-02-17T06:30:03.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542297' 2026-02-17T06:30:03.411 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542297 2026-02-17T06:30:03.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:03.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542297 -lt 60129542297 2026-02-17T06:30:03.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:03.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:03.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:03.719 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:03.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:03.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:03.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:03.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:03.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:03.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:03.881 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:03.881 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:03.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:03.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:03.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:03.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836651 2026-02-17T06:30:03.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836651 2026-02-17T06:30:03.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836651' 2026-02-17T06:30:03.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:03.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:04.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705828 2026-02-17T06:30:04.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705828 2026-02-17T06:30:04.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836651 1-38654705828' 2026-02-17T06:30:04.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:04.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:04.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542300 2026-02-17T06:30:04.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542300 2026-02-17T06:30:04.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836651 1-38654705828 2-60129542300' 2026-02-17T06:30:04.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:04.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836651 2026-02-17T06:30:04.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:04.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:04.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836651 2026-02-17T06:30:04.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:04.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836651 2026-02-17T06:30:04.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836651' 2026-02-17T06:30:04.098 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836651 2026-02-17T06:30:04.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:04.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836649 -lt 21474836651 2026-02-17T06:30:04.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:05.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:05.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:05.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836652 -lt 21474836651 2026-02-17T06:30:05.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:05.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705828 2026-02-17T06:30:05.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:05.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:05.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705828 2026-02-17T06:30:05.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:05.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705828 2026-02-17T06:30:05.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705828' 2026-02-17T06:30:05.410 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705828 2026-02-17T06:30:05.410 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:05.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705828 -lt 38654705828 2026-02-17T06:30:05.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:05.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542300 2026-02-17T06:30:05.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:05.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:05.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542300 2026-02-17T06:30:05.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:05.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542300 2026-02-17T06:30:05.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542300' 2026-02-17T06:30:05.567 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542300 2026-02-17T06:30:05.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:05.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542300 -lt 60129542300 2026-02-17T06:30:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:05.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:05.862 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:05.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:05.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:05.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:05.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:05.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:06.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:06.017 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:06.017 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:06.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:06.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:06.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:06.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836654 2026-02-17T06:30:06.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836654 2026-02-17T06:30:06.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836654' 2026-02-17T06:30:06.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:06.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:06.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705831 2026-02-17T06:30:06.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705831 2026-02-17T06:30:06.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836654 1-38654705831' 2026-02-17T06:30:06.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:06.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:06.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542303 2026-02-17T06:30:06.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542303 2026-02-17T06:30:06.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836654 1-38654705831 2-60129542303' 2026-02-17T06:30:06.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:06.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836654 2026-02-17T06:30:06.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:06.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:06.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836654 2026-02-17T06:30:06.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:06.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836654 2026-02-17T06:30:06.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836654' 2026-02-17T06:30:06.238 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836654 2026-02-17T06:30:06.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:06.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836652 -lt 21474836654 2026-02-17T06:30:06.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:07.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:07.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:07.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836655 -lt 21474836654 2026-02-17T06:30:07.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:07.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705831 2026-02-17T06:30:07.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:07.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:07.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705831 2026-02-17T06:30:07.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:07.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705831 2026-02-17T06:30:07.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705831' 2026-02-17T06:30:07.544 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705831 2026-02-17T06:30:07.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:07.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705831 -lt 38654705831 2026-02-17T06:30:07.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:07.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542303 2026-02-17T06:30:07.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:07.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:07.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542303 2026-02-17T06:30:07.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:07.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542303 2026-02-17T06:30:07.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542303' 2026-02-17T06:30:07.714 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542303 2026-02-17T06:30:07.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:07.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542303 -lt 60129542303 2026-02-17T06:30:07.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:07.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:07.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:08.008 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:08.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:08.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:08.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:08.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:08.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:08.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:08.164 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:08.164 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:08.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:08.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:08.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:08.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836657 2026-02-17T06:30:08.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836657 2026-02-17T06:30:08.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836657' 2026-02-17T06:30:08.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:08.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:08.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705834 2026-02-17T06:30:08.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705834 2026-02-17T06:30:08.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836657 1-38654705834' 2026-02-17T06:30:08.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:08.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:08.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542306 2026-02-17T06:30:08.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542306 2026-02-17T06:30:08.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836657 1-38654705834 2-60129542306' 2026-02-17T06:30:08.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:08.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836657 2026-02-17T06:30:08.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:08.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:08.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836657 2026-02-17T06:30:08.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:08.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836657 2026-02-17T06:30:08.380 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836657 2026-02-17T06:30:08.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836657' 2026-02-17T06:30:08.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:08.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836655 -lt 21474836657 2026-02-17T06:30:08.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:09.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:09.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:09.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836658 -lt 21474836657 2026-02-17T06:30:09.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:09.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705834 2026-02-17T06:30:09.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:09.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:09.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705834 2026-02-17T06:30:09.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:09.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705834 2026-02-17T06:30:09.684 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705834 2026-02-17T06:30:09.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705834' 2026-02-17T06:30:09.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:09.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705834 -lt 38654705834 2026-02-17T06:30:09.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:09.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542306 2026-02-17T06:30:09.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:09.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:09.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542306 2026-02-17T06:30:09.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:09.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542306 2026-02-17T06:30:09.842 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542306 2026-02-17T06:30:09.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542306' 2026-02-17T06:30:09.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:10.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542306 -lt 60129542306 2026-02-17T06:30:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:10.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:10.137 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:10.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:10.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:10.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:10.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:10.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:10.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:10.292 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:10.292 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:10.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:10.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:10.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:10.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836660 2026-02-17T06:30:10.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836660 2026-02-17T06:30:10.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836660' 2026-02-17T06:30:10.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:10.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:10.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705837 2026-02-17T06:30:10.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705837 2026-02-17T06:30:10.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836660 1-38654705837' 2026-02-17T06:30:10.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:10.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:10.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542309 2026-02-17T06:30:10.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542309 2026-02-17T06:30:10.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836660 1-38654705837 2-60129542309' 2026-02-17T06:30:10.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:10.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836660 2026-02-17T06:30:10.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:10.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:10.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836660 2026-02-17T06:30:10.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:10.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836660 2026-02-17T06:30:10.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836660' 2026-02-17T06:30:10.509 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836660 2026-02-17T06:30:10.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836661 -lt 21474836660 2026-02-17T06:30:10.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:10.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705837 2026-02-17T06:30:10.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:10.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:10.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705837 2026-02-17T06:30:10.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:10.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705837 2026-02-17T06:30:10.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705837' 2026-02-17T06:30:10.672 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705837 2026-02-17T06:30:10.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:10.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705837 -lt 38654705837 2026-02-17T06:30:10.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:10.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542309 2026-02-17T06:30:10.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:10.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:10.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542309 2026-02-17T06:30:10.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542309 2026-02-17T06:30:10.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542309' 2026-02-17T06:30:10.838 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542309 2026-02-17T06:30:10.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:10.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542309 -lt 60129542309 2026-02-17T06:30:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:10.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:11.129 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:11.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:11.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:11.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:11.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:11.139 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:11.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:11.292 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:11.292 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:11.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:11.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:11.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836662 2026-02-17T06:30:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836662 2026-02-17T06:30:11.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836662' 2026-02-17T06:30:11.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:11.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:11.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705839 2026-02-17T06:30:11.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705839 2026-02-17T06:30:11.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836662 1-38654705839' 2026-02-17T06:30:11.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:11.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:11.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542311 2026-02-17T06:30:11.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542311 2026-02-17T06:30:11.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836662 1-38654705839 2-60129542311' 2026-02-17T06:30:11.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:11.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836662 2026-02-17T06:30:11.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:11.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:11.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836662 2026-02-17T06:30:11.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:11.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836662 2026-02-17T06:30:11.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836662' 2026-02-17T06:30:11.513 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836662 2026-02-17T06:30:11.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:11.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836661 -lt 21474836662 2026-02-17T06:30:11.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:12.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:12.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:12.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836664 -lt 21474836662 2026-02-17T06:30:12.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:12.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705839 2026-02-17T06:30:12.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:12.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:12.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705839 2026-02-17T06:30:12.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:12.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705839 2026-02-17T06:30:12.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705839' 2026-02-17T06:30:12.828 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705839 2026-02-17T06:30:12.828 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:12.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705840 -lt 38654705839 2026-02-17T06:30:12.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:12.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542311 2026-02-17T06:30:12.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:12.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:12.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542311 2026-02-17T06:30:12.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:12.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542311 2026-02-17T06:30:12.990 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542311 2026-02-17T06:30:12.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542311' 2026-02-17T06:30:12.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:13.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542312 -lt 60129542311 2026-02-17T06:30:13.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:13.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:13.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:13.286 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:13.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:13.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:13.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:13.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:13.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:13.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:13.441 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:13.441 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:13.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:13.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:13.441 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:13.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836666 2026-02-17T06:30:13.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836666 2026-02-17T06:30:13.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836666' 2026-02-17T06:30:13.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:13.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:13.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705842 2026-02-17T06:30:13.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705842 2026-02-17T06:30:13.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836666 1-38654705842' 2026-02-17T06:30:13.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:13.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:13.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542315 2026-02-17T06:30:13.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542315 2026-02-17T06:30:13.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836666 1-38654705842 2-60129542315' 2026-02-17T06:30:13.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:13.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836666 2026-02-17T06:30:13.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:13.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:13.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836666 2026-02-17T06:30:13.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:13.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836666 2026-02-17T06:30:13.661 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836666 2026-02-17T06:30:13.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836666' 2026-02-17T06:30:13.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:13.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836664 -lt 21474836666 2026-02-17T06:30:13.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:14.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:14.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:14.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836667 -lt 21474836666 2026-02-17T06:30:14.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:14.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705842 2026-02-17T06:30:14.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:14.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:14.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705842 2026-02-17T06:30:14.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:14.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705842 2026-02-17T06:30:14.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705842' 2026-02-17T06:30:14.972 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705842 2026-02-17T06:30:14.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:15.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705843 -lt 38654705842 2026-02-17T06:30:15.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:15.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542315 2026-02-17T06:30:15.134 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:15.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:15.136 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542315 2026-02-17T06:30:15.136 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:15.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542315 2026-02-17T06:30:15.137 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542315 2026-02-17T06:30:15.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542315' 2026-02-17T06:30:15.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:15.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542315 -lt 60129542315 2026-02-17T06:30:15.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:15.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:15.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:15.442 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:15.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:15.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:15.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:15.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:15.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:15.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:15.600 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:15.600 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:15.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:15.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:15.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:15.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836669 2026-02-17T06:30:15.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836669 2026-02-17T06:30:15.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836669' 2026-02-17T06:30:15.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:15.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705845 2026-02-17T06:30:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705845 2026-02-17T06:30:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836669 1-38654705845' 2026-02-17T06:30:15.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:15.742 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:15.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542318 2026-02-17T06:30:15.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542318 2026-02-17T06:30:15.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836669 1-38654705845 2-60129542318' 2026-02-17T06:30:15.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:15.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836669 2026-02-17T06:30:15.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:15.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:15.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836669 2026-02-17T06:30:15.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:15.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836669 2026-02-17T06:30:15.815 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836669 2026-02-17T06:30:15.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836669' 2026-02-17T06:30:15.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:15.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836667 -lt 21474836669 2026-02-17T06:30:15.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:16.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:16.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:17.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836670 -lt 21474836669 2026-02-17T06:30:17.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:17.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705845 2026-02-17T06:30:17.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:17.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:17.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705845 2026-02-17T06:30:17.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:17.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705845 2026-02-17T06:30:17.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705845' 2026-02-17T06:30:17.130 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705845 2026-02-17T06:30:17.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:17.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705846 -lt 38654705845 2026-02-17T06:30:17.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:17.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542318 2026-02-17T06:30:17.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:17.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:17.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542318 2026-02-17T06:30:17.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:17.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542318 2026-02-17T06:30:17.283 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542318 2026-02-17T06:30:17.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542318' 2026-02-17T06:30:17.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:17.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542318 -lt 60129542318 2026-02-17T06:30:17.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:17.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:17.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:17.580 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:17.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:17.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:17.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:17.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:17.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:17.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:17.735 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:17.735 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:17.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:17.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:17.735 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:17.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836672 2026-02-17T06:30:17.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836672 2026-02-17T06:30:17.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836672' 2026-02-17T06:30:17.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:17.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:17.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705848 2026-02-17T06:30:17.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705848 2026-02-17T06:30:17.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836672 1-38654705848' 2026-02-17T06:30:17.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:17.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:17.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542321 2026-02-17T06:30:17.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542321 2026-02-17T06:30:17.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836672 1-38654705848 2-60129542321' 2026-02-17T06:30:17.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:17.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836672 2026-02-17T06:30:17.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:17.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:17.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836672 2026-02-17T06:30:17.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:17.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836672 2026-02-17T06:30:17.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836672' 2026-02-17T06:30:17.956 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836672 2026-02-17T06:30:17.956 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:18.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836670 -lt 21474836672 2026-02-17T06:30:18.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:19.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:19.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:19.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836673 -lt 21474836672 2026-02-17T06:30:19.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:19.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705848 2026-02-17T06:30:19.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:19.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:19.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705848 2026-02-17T06:30:19.262 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:19.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705848 2026-02-17T06:30:19.263 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705848 2026-02-17T06:30:19.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705848' 2026-02-17T06:30:19.264 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:19.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705849 -lt 38654705848 2026-02-17T06:30:19.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:19.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542321 2026-02-17T06:30:19.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:19.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:19.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542321 2026-02-17T06:30:19.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:19.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542321 2026-02-17T06:30:19.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542321' 2026-02-17T06:30:19.430 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542321 2026-02-17T06:30:19.430 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:19.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542321 -lt 60129542321 2026-02-17T06:30:19.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:19.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:19.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:19.717 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:19.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:19.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:19.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:19.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:19.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:19.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:19.877 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:19.877 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:19.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:19.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:19.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:19.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836675 2026-02-17T06:30:19.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836675 2026-02-17T06:30:19.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836675' 2026-02-17T06:30:19.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:19.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:20.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705852 2026-02-17T06:30:20.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705852 2026-02-17T06:30:20.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836675 1-38654705852' 2026-02-17T06:30:20.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:20.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:20.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542324 2026-02-17T06:30:20.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542324 2026-02-17T06:30:20.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836675 1-38654705852 2-60129542324' 2026-02-17T06:30:20.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:20.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836675 2026-02-17T06:30:20.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:20.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:20.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836675 2026-02-17T06:30:20.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:20.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836675 2026-02-17T06:30:20.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836675' 2026-02-17T06:30:20.094 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836675 2026-02-17T06:30:20.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:20.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836673 -lt 21474836675 2026-02-17T06:30:20.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:21.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:21.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:21.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836676 -lt 21474836675 2026-02-17T06:30:21.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:21.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705852 2026-02-17T06:30:21.399 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:21.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:21.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705852 2026-02-17T06:30:21.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:21.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705852 2026-02-17T06:30:21.402 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705852 2026-02-17T06:30:21.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705852' 2026-02-17T06:30:21.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:21.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705852 -lt 38654705852 2026-02-17T06:30:21.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:21.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542324 2026-02-17T06:30:21.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:21.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:21.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542324 2026-02-17T06:30:21.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:21.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542324 2026-02-17T06:30:21.569 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542324 2026-02-17T06:30:21.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542324' 2026-02-17T06:30:21.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:21.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542324 -lt 60129542324 2026-02-17T06:30:21.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:21.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:21.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:21.862 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:21.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:21.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:21.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:21.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:21.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:22.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:22.022 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:22.022 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:22.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:22.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:22.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:22.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836678 2026-02-17T06:30:22.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836678 2026-02-17T06:30:22.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836678' 2026-02-17T06:30:22.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:22.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:22.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705855 2026-02-17T06:30:22.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705855 2026-02-17T06:30:22.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836678 1-38654705855' 2026-02-17T06:30:22.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:22.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:22.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542327 2026-02-17T06:30:22.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542327 2026-02-17T06:30:22.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836678 1-38654705855 2-60129542327' 2026-02-17T06:30:22.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:22.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836678 2026-02-17T06:30:22.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:22.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:22.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836678 2026-02-17T06:30:22.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:22.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836678 2026-02-17T06:30:22.237 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836678 2026-02-17T06:30:22.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836678' 2026-02-17T06:30:22.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:22.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836676 -lt 21474836678 2026-02-17T06:30:22.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:23.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:23.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:23.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836679 -lt 21474836678 2026-02-17T06:30:23.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:23.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705855 2026-02-17T06:30:23.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:23.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:23.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705855 2026-02-17T06:30:23.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:23.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705855 2026-02-17T06:30:23.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705855' 2026-02-17T06:30:23.544 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705855 2026-02-17T06:30:23.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:23.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705855 -lt 38654705855 2026-02-17T06:30:23.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:23.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542327 2026-02-17T06:30:23.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:23.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:23.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542327 2026-02-17T06:30:23.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:23.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542327 2026-02-17T06:30:23.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542327' 2026-02-17T06:30:23.701 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542327 2026-02-17T06:30:23.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:23.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542327 -lt 60129542327 2026-02-17T06:30:23.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:23.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:23.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:23.990 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:23.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:23.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:23.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:23.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:24.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:24.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:24.151 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:24.151 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:24.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:24.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:24.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:24.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836681 2026-02-17T06:30:24.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836681 2026-02-17T06:30:24.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836681' 2026-02-17T06:30:24.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:24.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:24.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705858 2026-02-17T06:30:24.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705858 2026-02-17T06:30:24.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836681 1-38654705858' 2026-02-17T06:30:24.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:24.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542330 2026-02-17T06:30:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542330 2026-02-17T06:30:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836681 1-38654705858 2-60129542330' 2026-02-17T06:30:24.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:24.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836681 2026-02-17T06:30:24.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:24.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:24.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836681 2026-02-17T06:30:24.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:24.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836681 2026-02-17T06:30:24.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836681' 2026-02-17T06:30:24.367 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836681 2026-02-17T06:30:24.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:24.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836679 -lt 21474836681 2026-02-17T06:30:24.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:25.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:25.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836682 -lt 21474836681 2026-02-17T06:30:25.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:25.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705858 2026-02-17T06:30:25.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:25.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:25.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705858 2026-02-17T06:30:25.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:25.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705858 2026-02-17T06:30:25.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705858' 2026-02-17T06:30:25.686 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705858 2026-02-17T06:30:25.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:25.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705858 -lt 38654705858 2026-02-17T06:30:25.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542330 2026-02-17T06:30:25.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:25.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:25.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542330 2026-02-17T06:30:25.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:25.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542330 2026-02-17T06:30:25.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542330' 2026-02-17T06:30:25.844 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542330 2026-02-17T06:30:25.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:26.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542330 -lt 60129542330 2026-02-17T06:30:26.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:26.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:26.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:26.146 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:26.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:26.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:26.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:26.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:26.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:26.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:26.308 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:26.308 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:26.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:26.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:26.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:26.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836684 2026-02-17T06:30:26.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836684 2026-02-17T06:30:26.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836684' 2026-02-17T06:30:26.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:26.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:26.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705861 2026-02-17T06:30:26.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705861 2026-02-17T06:30:26.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836684 1-38654705861' 2026-02-17T06:30:26.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:26.451 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:26.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542333 2026-02-17T06:30:26.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542333 2026-02-17T06:30:26.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836684 1-38654705861 2-60129542333' 2026-02-17T06:30:26.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:26.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836684 2026-02-17T06:30:26.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:26.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:26.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836684 2026-02-17T06:30:26.524 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:26.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836684 2026-02-17T06:30:26.525 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836684 2026-02-17T06:30:26.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836684' 2026-02-17T06:30:26.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:26.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836685 -lt 21474836684 2026-02-17T06:30:26.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:26.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705861 2026-02-17T06:30:26.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:26.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:26.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705861 2026-02-17T06:30:26.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:26.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705861 2026-02-17T06:30:26.682 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705861 2026-02-17T06:30:26.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705861' 2026-02-17T06:30:26.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:26.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705861 -lt 38654705861 2026-02-17T06:30:26.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:26.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542333 2026-02-17T06:30:26.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:26.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:26.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542333 2026-02-17T06:30:26.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:26.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542333 2026-02-17T06:30:26.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542333' 2026-02-17T06:30:26.836 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542333 2026-02-17T06:30:26.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:26.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542333 -lt 60129542333 2026-02-17T06:30:26.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:26.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:26.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:27.143 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:27.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:27.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:27.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:27.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:27.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:27.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:27.298 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:27.298 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:27.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:27.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:27.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:27.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836686 2026-02-17T06:30:27.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836686 2026-02-17T06:30:27.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686' 2026-02-17T06:30:27.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:27.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:27.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705863 2026-02-17T06:30:27.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705863 2026-02-17T06:30:27.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686 1-38654705863' 2026-02-17T06:30:27.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:27.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542335 2026-02-17T06:30:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542335 2026-02-17T06:30:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686 1-38654705863 2-60129542335' 2026-02-17T06:30:27.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:27.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836686 2026-02-17T06:30:27.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:27.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:27.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836686 2026-02-17T06:30:27.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:27.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836686 2026-02-17T06:30:27.517 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836686 2026-02-17T06:30:27.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836686' 2026-02-17T06:30:27.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:27.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836685 -lt 21474836686 2026-02-17T06:30:27.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:28.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:28.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:28.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836688 -lt 21474836686 2026-02-17T06:30:28.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:28.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705863 2026-02-17T06:30:28.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:28.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:28.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705863 2026-02-17T06:30:28.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:28.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705863 2026-02-17T06:30:28.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705863' 2026-02-17T06:30:28.822 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705863 2026-02-17T06:30:28.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:28.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705864 -lt 38654705863 2026-02-17T06:30:28.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:28.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542335 2026-02-17T06:30:28.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:28.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:28.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542335 2026-02-17T06:30:28.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:28.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542335 2026-02-17T06:30:28.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542335' 2026-02-17T06:30:28.977 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542335 2026-02-17T06:30:28.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:29.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542336 -lt 60129542335 2026-02-17T06:30:29.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:29.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:29.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:29.278 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:29.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:29.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:29.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:29.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:29.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:29.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:29.434 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:29.434 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:29.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:29.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:29.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836690 2026-02-17T06:30:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836690 2026-02-17T06:30:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836690' 2026-02-17T06:30:29.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:29.506 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:29.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705866 2026-02-17T06:30:29.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705866 2026-02-17T06:30:29.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836690 1-38654705866' 2026-02-17T06:30:29.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:29.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:29.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542339 2026-02-17T06:30:29.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542339 2026-02-17T06:30:29.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836690 1-38654705866 2-60129542339' 2026-02-17T06:30:29.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:29.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836690 2026-02-17T06:30:29.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:29.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:29.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836690 2026-02-17T06:30:29.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:29.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836690 2026-02-17T06:30:29.650 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836690 2026-02-17T06:30:29.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836690' 2026-02-17T06:30:29.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:29.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836688 -lt 21474836690 2026-02-17T06:30:29.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:30.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:30.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:30.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836691 -lt 21474836690 2026-02-17T06:30:30.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:30.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705866 2026-02-17T06:30:30.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:30.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:30.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705866 2026-02-17T06:30:30.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:30.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705866 2026-02-17T06:30:30.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705866' 2026-02-17T06:30:30.968 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705866 2026-02-17T06:30:30.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:31.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705867 -lt 38654705866 2026-02-17T06:30:31.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:31.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542339 2026-02-17T06:30:31.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:31.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:31.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542339 2026-02-17T06:30:31.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:31.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542339 2026-02-17T06:30:31.129 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542339 2026-02-17T06:30:31.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542339' 2026-02-17T06:30:31.130 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:31.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542339 -lt 60129542339 2026-02-17T06:30:31.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:31.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:31.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:31.423 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:31.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:31.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:31.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:31.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:31.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:31.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:31.574 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:31.574 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:31.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:31.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:31.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:31.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836693 2026-02-17T06:30:31.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836693 2026-02-17T06:30:31.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836693' 2026-02-17T06:30:31.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:31.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:31.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705869 2026-02-17T06:30:31.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705869 2026-02-17T06:30:31.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836693 1-38654705869' 2026-02-17T06:30:31.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:31.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542342 2026-02-17T06:30:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542342 2026-02-17T06:30:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836693 1-38654705869 2-60129542342' 2026-02-17T06:30:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:31.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836693 2026-02-17T06:30:31.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:31.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:31.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836693 2026-02-17T06:30:31.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:31.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836693 2026-02-17T06:30:31.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836693' 2026-02-17T06:30:31.794 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836693 2026-02-17T06:30:31.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:31.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836691 -lt 21474836693 2026-02-17T06:30:31.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:32.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:32.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:33.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836694 -lt 21474836693 2026-02-17T06:30:33.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:33.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705869 2026-02-17T06:30:33.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:33.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:33.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705869 2026-02-17T06:30:33.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:33.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705869 2026-02-17T06:30:33.100 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705869 2026-02-17T06:30:33.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705869' 2026-02-17T06:30:33.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:33.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705870 -lt 38654705869 2026-02-17T06:30:33.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:33.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542342 2026-02-17T06:30:33.255 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:33.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:33.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542342 2026-02-17T06:30:33.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:33.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542342 2026-02-17T06:30:33.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542342' 2026-02-17T06:30:33.259 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542342 2026-02-17T06:30:33.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:33.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542342 -lt 60129542342 2026-02-17T06:30:33.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:33.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:33.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:33.549 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:33.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:33.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:33.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:33.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:33.560 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:33.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:33.709 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:33.710 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:33.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:33.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:33.710 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:33.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836696 2026-02-17T06:30:33.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836696 2026-02-17T06:30:33.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836696' 2026-02-17T06:30:33.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:33.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:33.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705872 2026-02-17T06:30:33.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705872 2026-02-17T06:30:33.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836696 1-38654705872' 2026-02-17T06:30:33.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:33.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:33.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542345 2026-02-17T06:30:33.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542345 2026-02-17T06:30:33.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836696 1-38654705872 2-60129542345' 2026-02-17T06:30:33.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:33.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836696 2026-02-17T06:30:33.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:33.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:33.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836696 2026-02-17T06:30:33.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:33.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836696 2026-02-17T06:30:33.926 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836696 2026-02-17T06:30:33.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836696' 2026-02-17T06:30:33.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:34.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836694 -lt 21474836696 2026-02-17T06:30:34.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:35.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:35.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:35.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836697 -lt 21474836696 2026-02-17T06:30:35.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:35.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705872 2026-02-17T06:30:35.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:35.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:35.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705872 2026-02-17T06:30:35.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:35.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705872 2026-02-17T06:30:35.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705872' 2026-02-17T06:30:35.240 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705872 2026-02-17T06:30:35.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:35.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705873 -lt 38654705872 2026-02-17T06:30:35.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:35.404 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542345 2026-02-17T06:30:35.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:35.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:35.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542345 2026-02-17T06:30:35.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:35.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542345 2026-02-17T06:30:35.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542345' 2026-02-17T06:30:35.408 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542345 2026-02-17T06:30:35.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:35.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542345 -lt 60129542345 2026-02-17T06:30:35.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:35.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:35.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:35.697 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:35.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:35.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:35.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:35.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:35.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:35.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:35.857 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:35.857 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:35.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:35.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:35.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:35.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836699 2026-02-17T06:30:35.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836699 2026-02-17T06:30:35.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836699' 2026-02-17T06:30:35.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:35.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:36.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705876 2026-02-17T06:30:36.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705876 2026-02-17T06:30:36.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836699 1-38654705876' 2026-02-17T06:30:36.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:36.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:36.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542348 2026-02-17T06:30:36.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542348 2026-02-17T06:30:36.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836699 1-38654705876 2-60129542348' 2026-02-17T06:30:36.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:36.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836699 2026-02-17T06:30:36.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:36.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:36.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836699 2026-02-17T06:30:36.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:36.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836699 2026-02-17T06:30:36.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836699' 2026-02-17T06:30:36.078 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836699 2026-02-17T06:30:36.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:36.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836697 -lt 21474836699 2026-02-17T06:30:36.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:37.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:37.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:37.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836700 -lt 21474836699 2026-02-17T06:30:37.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:37.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705876 2026-02-17T06:30:37.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:37.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:37.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705876 2026-02-17T06:30:37.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:37.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705876 2026-02-17T06:30:37.386 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705876 2026-02-17T06:30:37.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705876' 2026-02-17T06:30:37.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:37.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705876 -lt 38654705876 2026-02-17T06:30:37.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:37.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542348 2026-02-17T06:30:37.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:37.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:37.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542348 2026-02-17T06:30:37.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:37.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542348 2026-02-17T06:30:37.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542348' 2026-02-17T06:30:37.549 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542348 2026-02-17T06:30:37.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:37.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542348 -lt 60129542348 2026-02-17T06:30:37.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:37.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:37.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:37.848 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:37.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:37.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:37.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:37.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:37.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:38.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:38.002 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:38.002 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:38.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:38.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:38.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:38.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836702 2026-02-17T06:30:38.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836702 2026-02-17T06:30:38.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836702' 2026-02-17T06:30:38.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:38.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:38.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705879 2026-02-17T06:30:38.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705879 2026-02-17T06:30:38.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836702 1-38654705879' 2026-02-17T06:30:38.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:38.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:38.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542351 2026-02-17T06:30:38.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542351 2026-02-17T06:30:38.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836702 1-38654705879 2-60129542351' 2026-02-17T06:30:38.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:38.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836702 2026-02-17T06:30:38.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:38.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:38.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836702 2026-02-17T06:30:38.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:38.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836702 2026-02-17T06:30:38.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836702' 2026-02-17T06:30:38.222 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836702 2026-02-17T06:30:38.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836700 -lt 21474836702 2026-02-17T06:30:38.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:39.369 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:39.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:39.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836703 -lt 21474836702 2026-02-17T06:30:39.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:39.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705879 2026-02-17T06:30:39.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:39.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:39.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705879 2026-02-17T06:30:39.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:39.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705879 2026-02-17T06:30:39.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705879' 2026-02-17T06:30:39.531 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705879 2026-02-17T06:30:39.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:39.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705879 -lt 38654705879 2026-02-17T06:30:39.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:39.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542351 2026-02-17T06:30:39.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:39.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:39.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542351 2026-02-17T06:30:39.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:39.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542351 2026-02-17T06:30:39.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542351' 2026-02-17T06:30:39.693 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542351 2026-02-17T06:30:39.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:39.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542351 -lt 60129542351 2026-02-17T06:30:39.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:39.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:39.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:39.987 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:39.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:30:39.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:30:39.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:30:39.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:30:39.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:30:40.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:30:40.144 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:30:40.144 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:30:40.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:30:40.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:40.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:30:40.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836705 2026-02-17T06:30:40.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836705 2026-02-17T06:30:40.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836705' 2026-02-17T06:30:40.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:40.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:30:40.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705882 2026-02-17T06:30:40.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705882 2026-02-17T06:30:40.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836705 1-38654705882' 2026-02-17T06:30:40.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:30:40.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:30:40.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542354 2026-02-17T06:30:40.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542354 2026-02-17T06:30:40.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836705 1-38654705882 2-60129542354' 2026-02-17T06:30:40.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:40.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836705 2026-02-17T06:30:40.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:40.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:30:40.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836705 2026-02-17T06:30:40.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:40.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836705 2026-02-17T06:30:40.358 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836705 2026-02-17T06:30:40.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836705' 2026-02-17T06:30:40.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:40.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836703 -lt 21474836705 2026-02-17T06:30:40.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:30:41.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:30:41.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:30:41.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836706 -lt 21474836705 2026-02-17T06:30:41.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:41.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705882 2026-02-17T06:30:41.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:41.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:30:41.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705882 2026-02-17T06:30:41.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:41.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705882 2026-02-17T06:30:41.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705882' 2026-02-17T06:30:41.671 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705882 2026-02-17T06:30:41.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:30:41.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705882 -lt 38654705882 2026-02-17T06:30:41.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:30:41.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542354 2026-02-17T06:30:41.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:30:41.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:30:41.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542354 2026-02-17T06:30:41.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:30:41.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542354 2026-02-17T06:30:41.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542354' 2026-02-17T06:30:41.834 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542354 2026-02-17T06:30:41.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:30:41.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542354 -lt 60129542354 2026-02-17T06:30:41.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:30:41.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:30:41.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:30:42.126 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:30:42.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:383: _scrub_abort: break 2026-02-17T06:30:42.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:385: _scrub_abort: set +o pipefail 2026-02-17T06:30:42.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:387: _scrub_abort: sleep 5 2026-02-17T06:30:47.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:389: _scrub_abort: grep 'nodeep_scrub set, aborting' td/osd-scrub-test/osd.1.log 2026-02-17T06:30:47.147 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:28:43.020+0000 7ff51745c640 10 osd.1 pg_epoch: 19 pg[1.0( v 17'1000 (0'0,17'1000] local-lis/les=15/16 n=1000 ec=15/15 lis/c=15/15 les/c/f=16/16/0 sis=15) [1,0,2] r=0 lpr=15 crt=17'1000 lcod 17'999 mlcod 17'999 active+clean+scrubbing+deep [ 1.0: ] TIME_FOR_DEEP] scrubber: nodeep_scrub set, aborting 2026-02-17T06:30:47.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: get_last_scrub_stamp 1.0 2026-02-17T06:30:47.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:47.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:47.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:47.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:47.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: local last_scrub=2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:47.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:396: _scrub_abort: ceph config set osd osd_scrub_sleep 0.1 2026-02-17T06:30:47.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:398: _scrub_abort: ceph osd unset nodeep-scrub 2026-02-17T06:30:47.642 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is unset 2026-02-17T06:30:47.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:399: _scrub_abort: '[' deep-scrub = deep-scrub ']' 2026-02-17T06:30:47.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:401: _scrub_abort: ceph osd unset noscrub 2026-02-17T06:30:47.843 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is unset 2026-02-17T06:30:47.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:403: _scrub_abort: TIMEOUT=500 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:404: _scrub_abort: wait_for_scrub 1.0 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:47.855 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:47.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:47.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:48.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:48.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:49.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:49.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:49.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:49.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:49.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:49.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:49.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:49.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:49.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:50.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:50.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:50.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:50.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:50.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:50.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:50.159 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:50.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:51.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:51.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:51.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:51.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:51.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:51.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:51.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:52.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:52.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:52.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:52.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:52.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:52.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:52.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:52.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:52.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:53.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:53.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:53.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:53.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:53.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:53.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:53.617 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:54.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:54.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:54.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:54.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:54.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:54.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:54.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:54.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:54.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:55.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:55.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:55.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:55.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:55.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:55.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:55.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:56.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:56.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:57.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:57.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:57.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:57.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:57.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:57.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:57.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:57.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:57.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:58.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:58.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:58.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:58.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:58.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:58.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:58.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:58.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:58.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:30:59.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:30:59.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:30:59.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:30:59.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:30:59.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:30:59.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:30:59.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:30:59.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:30:59.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:00.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:00.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:00.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:00.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:00.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:00.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:00.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:00.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:00.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:01.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:01.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:01.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:01.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:01.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:01.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:01.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:01.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:01.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:02.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:02.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:02.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:02.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:02.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:02.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:02.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:02.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:02.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:03.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:03.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:03.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:03.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:03.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:03.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:03.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:04.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:04.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:05.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:05.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:05.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:05.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:05.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:05.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:05.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:05.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:05.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:06.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:06.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:06.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:06.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:06.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:06.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:06.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:06.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:06.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:07.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:07.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:07.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:07.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:07.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:07.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:07.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:07.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:07.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:08.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:08.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:08.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:08.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:08.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:08.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:08.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:08.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:08.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:09.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:09.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:09.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:09.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:09.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:09.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:09.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:09.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:10.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:10.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:10.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:10.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:10.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:10.882 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:10.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:11.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:11.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:12.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:12.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:12.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:12.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:12.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:12.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:12.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:12.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:12.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:13.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:13.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:13.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:13.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:13.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:13.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:13.193 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:13.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:13.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:14.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:14.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:14.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:14.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:14.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:14.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:14.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:14.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:14.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:15.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:15.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:15.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:15.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:15.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:15.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:15.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:15.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:15.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:16.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:16.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:16.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:16.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:16.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:16.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:16.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:16.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:16.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:17.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:17.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:17.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:17.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:17.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:17.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:17.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:17.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:17.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:18.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:18.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:18.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:18.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:18.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:18.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:18.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:19.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:19.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:20.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:20.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:20.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:20.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:20.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:20.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:20.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:20.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:20.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:21.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:21.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:21.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:21.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:21.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:21.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:21.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:21.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:21.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:22.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:22.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:22.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:22.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:22.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:22.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:22.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:22.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:22.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:23.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:23.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:23.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:23.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:23.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:23.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:23.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:23.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:23.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:24.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:24.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:24.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:24.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:24.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:24.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:24.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:24.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:24.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:25.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:25.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:25.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:25.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:25.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:25.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:25.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:26.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:26.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:27.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:27.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:27.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:27.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:27.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:27.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:27.026 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:27.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:27.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:28.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:28.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:28.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:28.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:28.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:28.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:28.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:28.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:28.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:29.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:29.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:29.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:29.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:29.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:29.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:29.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:29.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:29.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:30.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:30.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:30.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:30.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:30.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:30.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:30.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:30.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:30.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:31.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:31.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:31.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:31.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:31.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:31.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:31.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:31.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:31.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:32.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:32.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:32.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:32.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:32.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:32.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:32.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:32.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:32.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:33.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:33.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:33.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:33.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:33.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:33.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:34.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:34.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:35.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:35.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:35.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:35.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:35.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:35.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:35.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:35.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:35.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:36.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:36.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:36.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:36.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:36.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:36.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:36.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:36.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:36.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:37.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:37.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:37.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:37.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:37.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:37.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:37.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:37.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:37.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:38.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:38.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:38.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:38.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:38.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:38.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:38.532 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:38.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:38.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:39.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:39.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:39.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:39.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:39.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:39.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:39.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:39.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:39.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:40.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:40.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:40.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:40.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:40.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:40.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:40.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:40.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:40.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:41.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:41.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:41.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:41.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:41.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:41.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:41.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:42.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:42.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:43.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:43.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:43.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:43.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:43.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:43.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:43.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:43.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:43.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:44.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:44.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:44.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:44.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:44.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:44.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:44.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:44.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:44.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:45.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:45.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:45.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:45.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:45.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:45.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:45.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:45.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:45.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:46.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:46.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:46.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:46.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:46.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:46.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:46.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:46.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:46.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:47.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:47.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:47.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:47.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:47.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:47.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:47.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:47.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:47.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:48.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:48.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:48.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:48.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:48.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:48.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:48.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:49.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:49.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:50.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:50.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:50.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:50.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:50.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:50.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:50.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:50.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:50.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:51.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:51.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:51.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:51.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:51.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:51.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:51.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:51.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:51.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:52.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:52.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:52.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:52.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:52.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:52.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:52.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:52.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:52.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:53.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:53.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:53.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:53.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:53.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:53.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:53.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:53.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:53.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:54.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:54.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:54.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:54.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:54.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:54.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:54.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:54.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:54.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:55.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:55.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:55.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:55.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:55.820 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:55.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:55.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:55.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:55.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:56.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:56.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:56.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:56.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:56.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:56.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:56.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:57.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:57.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:58.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:58.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:58.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:58.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:58.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:58.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:58.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:58.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:58.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:31:59.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:31:59.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:31:59.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:31:59.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:31:59.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:31:59.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:31:59.266 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:31:59.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:31:59.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:00.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:00.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:00.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:00.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:00.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:00.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:00.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:00.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:00.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:01.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:01.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:01.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:01.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:01.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:01.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:01.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:01.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:01.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:02.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:02.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:02.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:02.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:02.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:02.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:02.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:02.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:02.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:03.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:03.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:03.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:03.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:03.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:03.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:03.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:04.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:04.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:05.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:05.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:05.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:05.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:05.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:05.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:05.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:05.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:05.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:06.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:06.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:06.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:06.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:06.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:06.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:06.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:06.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:06.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:07.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:07.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:07.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:07.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:07.321 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:07.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:07.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:07.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:07.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:08.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:08.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:08.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:08.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:08.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:08.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:08.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:08.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:08.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:09.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:09.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:09.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:09.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:09.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:09.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:09.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:09.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:09.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:10.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:10.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:10.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:10.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:10.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:10.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:10.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:10.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:10.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:11.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:11.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:11.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:11.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:11.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:11.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:11.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:13.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:13.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:13.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:13.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:13.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:13.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:13.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:13.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:13.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:14.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:14.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:14.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:14.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:14.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:14.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:14.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:14.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:14.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:15.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:15.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:15.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:15.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:15.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:15.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:15.375 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:15.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:15.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:16.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:16.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:16.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:16.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:16.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:16.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:16.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:16.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:16.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:17.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:17.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:17.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:17.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:17.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:17.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:17.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:17.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:17.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:18.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:18.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:18.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:18.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:18.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:18.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:18.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:19.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:19.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:20.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:20.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:21.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:21.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:21.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:21.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:21.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:21.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:21.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:21.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:21.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:22.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:22.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:22.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:22.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:22.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:22.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:22.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:22.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:22.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:23.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:23.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:23.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:23.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:23.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:23.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:23.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:23.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:23.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:24.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:24.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:24.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:24.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:24.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:24.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:24.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:24.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:24.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:25.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:25.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:25.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:25.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:25.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:25.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:25.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:25.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:25.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:26.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:26.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:26.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:26.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:26.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:26.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:26.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:27.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:27.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:28.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:28.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:28.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:28.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:28.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:28.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:28.060 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:28.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:28.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:29.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:29.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:29.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:29.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:29.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:29.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:29.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:29.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:29.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:30.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:30.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:30.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:30.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:30.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:30.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:30.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:30.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:30.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:31.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:31.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:31.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:31.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:31.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:31.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:31.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:31.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:31.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:32.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:32.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:32.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:32.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:32.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:32.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:32.670 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:32.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:32.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:33.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:33.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:33.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:33.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:33.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:33.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:33.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:34.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:34.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:34.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:34.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:34.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:34.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:34.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:35.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:35.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:36.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:36.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:36.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:36.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:36.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:36.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:36.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:36.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:36.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:37.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:37.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:37.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:37.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:37.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:37.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:37.281 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:37.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:37.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:38.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:38.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:38.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:38.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:38.434 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:38.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:38.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:38.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:38.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:39.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:39.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:39.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:39.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:39.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:39.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:39.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:39.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:39.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:40.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:40.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:40.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:40.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:40.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:40.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:40.738 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:40.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:40.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:41.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:41.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:41.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:41.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:41.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:41.893 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:41.893 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:42.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:42.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:43.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:43.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:43.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:43.044 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:43.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:43.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:43.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:43.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:43.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:44.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:44.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:44.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:44.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:44.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:44.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:44.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:44.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:44.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:45.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:45.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:45.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:45.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:45.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:45.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:45.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:45.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:45.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:46.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:46.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:46.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:46.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:46.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:46.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:46.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:46.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:46.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:47.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:47.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:47.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:47.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:47.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:47.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:47.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:47.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:47.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:48.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:48.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:48.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:48.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:48.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:48.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:48.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:48.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:48.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:49.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:49.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:49.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:49.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:49.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:49.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:49.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:50.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:50.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:51.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:51.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:51.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:51.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:51.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:51.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:51.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:51.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:52.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:52.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:52.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:52.251 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:52.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:52.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:52.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:52.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:52.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:53.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:53.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:53.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:53.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:53.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:53.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:53.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:53.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:53.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:54.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:54.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:54.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:54.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:54.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:54.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:54.557 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:54.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:54.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:55.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:55.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:55.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:55.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:55.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:55.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:55.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:55.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:55.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:56.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:56.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:56.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:56.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:56.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:56.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:56.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:57.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:57.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:58.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:58.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:58.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:58.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:58.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:58.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:58.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:58.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:58.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:32:59.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:32:59.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:32:59.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:32:59.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:32:59.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:32:59.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:32:59.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:32:59.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:32:59.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:00.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:00.309 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:00.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:00.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:00.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:00.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:00.310 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:00.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:00.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:01.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:01.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:01.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:01.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:01.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:01.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:01.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:01.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:01.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:02.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:02.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:02.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:02.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:02.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:02.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:02.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:02.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:02.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:03.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:03.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:03.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:03.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:03.753 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:03.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:03.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:03.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:03.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:04.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:04.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:04.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:04.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:04.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:04.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:04.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:05.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:05.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:06.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:06.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:06.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:06.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:06.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:06.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:06.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:06.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:06.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:07.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:07.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:07.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:07.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:07.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:07.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:07.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:07.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:07.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:08.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:08.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:08.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:08.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:08.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:08.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:08.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:08.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:08.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:09.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:09.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:09.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:09.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:09.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:09.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:09.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:09.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:09.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:10.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:10.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:10.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:10.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:10.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:10.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:10.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:10.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:10.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:11.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:11.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:11.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:11.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:11.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:11.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:11.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:11.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:11.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:12.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:12.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:12.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:12.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:12.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:12.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:12.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:13.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:13.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:14.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:14.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:14.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:14.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:14.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:14.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:14.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:14.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:14.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:15.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:15.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:15.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:15.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:15.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:15.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:15.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:15.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:15.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:16.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:16.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:16.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:16.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:16.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:16.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:16.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:16.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:16.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:17.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:17.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:17.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:17.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:17.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:17.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:17.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:17.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:17.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:18.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:18.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:18.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:18.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:18.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:18.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:18.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:18.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:18.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:19.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:19.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:19.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:19.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:19.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:19.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:19.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:19.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:19.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:20.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:20.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:20.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:20.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:20.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:20.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:20.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:21.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:21.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:22.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:22.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:22.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:22.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:22.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:22.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:22.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:22.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:22.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:23.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:23.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:23.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:23.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:23.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:23.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:23.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:23.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:23.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:24.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:24.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:24.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:24.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:24.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:24.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:24.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:24.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:24.586 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:25.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:25.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:25.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:25.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:25.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:25.588 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:25.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:25.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:25.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:26.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:26.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:26.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:26.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:26.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:26.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:26.741 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:26.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:26.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:27.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:27.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:27.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:27.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:27.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:27.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:27.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:28.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:28.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:29.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:29.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:29.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:29.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:29.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:29.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:29.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:29.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:29.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:30.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:30.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:30.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:30.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:30.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:30.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:30.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:30.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:30.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:31.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:31.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:31.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:31.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:31.355 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:31.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:31.356 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:32.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:32.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:32.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:32.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:32.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:32.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:32.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:32.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:32.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:33.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:33.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:33.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:33.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:33.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:33.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:33.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:33.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:33.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:34.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:34.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:34.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:34.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:34.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:34.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:34.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:34.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:34.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:35.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:35.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:35.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:35.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:35.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:35.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:35.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:36.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:36.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:37.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:37.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:37.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:37.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:37.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:37.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:37.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:37.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:37.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:38.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:38.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:38.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:38.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:38.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:38.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:38.269 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:38.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:38.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:39.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:39.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:39.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:39.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:39.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:39.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:39.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:39.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:39.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:40.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:40.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:40.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:40.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:40.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:40.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:40.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:40.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:40.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:41.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:41.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:41.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:41.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:41.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:41.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:41.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:41.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:41.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:42.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:42.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:42.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:42.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:42.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:42.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:42.873 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:43.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:43.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:44.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:44.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:44.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:44.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:44.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:44.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:44.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:45.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:45.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:45.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:45.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:45.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:45.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:45.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:45.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:45.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:46.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:46.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:46.323 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:46.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:46.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:46.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:46.324 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:46.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:46.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:47.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:47.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:47.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:47.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:47.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:47.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:47.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:47.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:47.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:48.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:48.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:48.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:48.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:48.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:48.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:48.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:48.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:48.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:49.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:49.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:49.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:49.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:49.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:49.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:49.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:49.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:49.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:50.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:50.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:50.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:50.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:50.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:50.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:50.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:51.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:51.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:52.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:52.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:52.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:52.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:52.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:52.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:52.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:52.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:52.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:53.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:53.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:53.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:53.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:53.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:53.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:53.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:53.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:53.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:54.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:54.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:54.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:54.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:54.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:54.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:54.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:54.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:54.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:55.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:55.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:55.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:55.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:55.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:55.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:55.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:55.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:55.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:56.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:56.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:56.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:56.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:56.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:56.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:56.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:56.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:56.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:57.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:57.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:57.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:57.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:57.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:57.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:57.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:57.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:57.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:33:58.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:33:58.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:33:58.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:33:58.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:33:58.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:33:58.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:33:58.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:33:59.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:33:59.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:00.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:00.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:00.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:00.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:00.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:00.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:00.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:00.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:00.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:01.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:01.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:01.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:01.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:01.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:01.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:01.306 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:01.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:01.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:02.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:02.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:02.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:02.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:02.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:02.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:02.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:02.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:02.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:03.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:03.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:03.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:03.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:03.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:03.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:03.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:03.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:04.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:04.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:04.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:04.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:04.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:04.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:04.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:04.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:04.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:05.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:05.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:05.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:05.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:05.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:05.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:05.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:06.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:06.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:07.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:07.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:07.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:07.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:07.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:07.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:07.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:07.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:07.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:08.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:08.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:08.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:08.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:08.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:08.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:08.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:08.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:08.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:09.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:09.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:09.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:09.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:09.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:09.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:09.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:09.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:09.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:10.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:10.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:10.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:10.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:10.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:10.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:10.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:10.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:10.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:11.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:11.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:11.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:11.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:11.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:11.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:11.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:11.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:11.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:12.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:12.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:12.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:12.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:12.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:12.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:12.832 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:12.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:12.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:13.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:13.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:13.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:13.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:13.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:13.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:13.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:14.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:14.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:15.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:15.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:15.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:15.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:15.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:15.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:15.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:15.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:15.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:16.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:16.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:16.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:16.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:16.286 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:16.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:16.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:16.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:16.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:17.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:17.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:17.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:17.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:17.443 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:17.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:17.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:17.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:17.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:18.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:18.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:18.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:18.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:18.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:18.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:18.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:18.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:18.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:19.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:19.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:19.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:19.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:19.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:19.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:19.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:19.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:19.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:20.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:20.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:20.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:20.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:20.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:20.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:20.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:21.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:21.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:22.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:22.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:22.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:22.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:22.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:22.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:22.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:22.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:23.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:23.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:23.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:23.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:23.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:23.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:23.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:23.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:23.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:24.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:24.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:24.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:24.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:24.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:24.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:24.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:24.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:24.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:25.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:25.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:25.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:25.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:25.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:25.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:25.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:25.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:26.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:26.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:26.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:26.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:26.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:26.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:26.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:26.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:27.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:27.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:27.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:27.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:27.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:27.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:27.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:27.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:27.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:28.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:28.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:28.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:28.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:28.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:28.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:28.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:29.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:29.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:30.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:30.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:30.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:30.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:30.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:30.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:30.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:30.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:30.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:31.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:31.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:31.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:31.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:31.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:31.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:31.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:31.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:31.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:32.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:32.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:32.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:32.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:32.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:32.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:32.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:32.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:32.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:33.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:33.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:33.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:33.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:33.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:33.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:33.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:33.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:33.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:34.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:34.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:34.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:34.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:34.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:34.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:34.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:34.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:34.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:35.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:35.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:35.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:35.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:35.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:35.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:35.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:35.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:35.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:36.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:36.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:36.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:36.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:36.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:37.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:37.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:37.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:37.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:38.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:38.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:38.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:38.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:38.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:38.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:38.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:38.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:38.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:39.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:39.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:39.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:39.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:39.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:39.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:39.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:39.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:39.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:40.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:40.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:40.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:40.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:40.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:40.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:40.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:40.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:40.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:41.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:41.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:41.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:41.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:41.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:41.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:41.610 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:41.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:41.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:42.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:42.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:42.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:42.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:42.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:42.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:42.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:42.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:42.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:43.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:43.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:43.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:43.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:43.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:43.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:43.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:44.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:44.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:45.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:45.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:45.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:45.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:45.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:45.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:45.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:45.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:45.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:46.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:46.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:46.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:46.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:46.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:46.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:46.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:46.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:46.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:47.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:47.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:47.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:47.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:47.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:47.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:47.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:47.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:47.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:48.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:48.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:48.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:48.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:48.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:48.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:48.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:48.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:48.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:49.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:49.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:49.663 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:49.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:49.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:49.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:49.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:49.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:49.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:50.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:50.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:50.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:50.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:50.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:50.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:50.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:50.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:50.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:51.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:51.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:51.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:51.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:51.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:51.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:51.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:52.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:52.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:53.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:53.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:53.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:53.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:53.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:53.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:53.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:53.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:53.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:54.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:54.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:54.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:54.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:54.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:54.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:54.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:54.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:54.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:55.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:55.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:55.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:55.420 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:55.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:55.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:55.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:55.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:55.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:56.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:56.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:56.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:56.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:56.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:56.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:56.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:56.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:56.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:57.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:57.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:57.718 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:57.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:57.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:57.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:57.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:57.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:57.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:34:58.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:34:58.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:34:58.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:34:58.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:34:58.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:34:58.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:34:58.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:34:59.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:34:59.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:00.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:00.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:00.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:00.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:00.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:00.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:00.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:00.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:00.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:01.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:01.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:01.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:01.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:01.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:01.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:01.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:01.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:01.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:02.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:02.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:02.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:02.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:02.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:02.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:02.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:02.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:02.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:03.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:03.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:03.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:03.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:03.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:03.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:03.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:03.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:03.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:04.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:04.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:04.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:04.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:04.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:04.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:04.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:04.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:04.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:05.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:05.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:05.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:05.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:05.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:05.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:05.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:05.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:05.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:06.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:06.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:06.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:06.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:06.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:06.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:06.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:07.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:07.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:08.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:08.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:08.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:08.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:08.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:08.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:08.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:08.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:08.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:09.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:09.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:09.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:09.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:09.215 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:09.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:09.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:09.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:09.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:10.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:10.368 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:10.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:10.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:10.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:10.369 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:10.370 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:10.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:10.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:11.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:11.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:11.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:11.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:11.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:11.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:11.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:11.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:11.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:12.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:12.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:12.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:12.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:12.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:12.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:12.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:12.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:12.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:13.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:13.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:13.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:13.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:13.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:13.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:13.823 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:13.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:13.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:14.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:14.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:15.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:15.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:16.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:16.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:16.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:16.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:16.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:16.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:16.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:16.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:16.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:17.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:17.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:17.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:17.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:17.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:17.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:17.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:17.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:17.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:18.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:18.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:18.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:18.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:18.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:18.426 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:18.427 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:18.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:18.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:19.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:19.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:19.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:19.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:19.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:19.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:19.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:19.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:19.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:20.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:20.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:20.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:20.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:20.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:20.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:20.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:20.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:20.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:21.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:21.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:21.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:21.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:21.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:21.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:21.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:22.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:22.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:23.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:23.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:23.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:23.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:23.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:23.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:23.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:23.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:23.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:24.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:24.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:24.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:24.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:24.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:24.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:24.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:24.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:24.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:25.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:25.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:25.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:25.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:25.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:25.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:25.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:25.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:25.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:26.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:26.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:26.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:26.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:26.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:26.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:26.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:26.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:26.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:27.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:27.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:27.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:27.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:27.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:27.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:27.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:27.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:27.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:28.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:28.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:28.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:28.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:28.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:28.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:28.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:28.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:28.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:29.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:29.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:29.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:29.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:29.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:29.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:29.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:30.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:30.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:31.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:31.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:31.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:31.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:31.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:31.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:31.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:31.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:31.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:32.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:32.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:32.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:32.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:32.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:32.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:32.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:32.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:32.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:33.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:33.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:33.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:33.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:33.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:33.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:33.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:33.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:33.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:34.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:34.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:34.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:34.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:34.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:34.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:34.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:34.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:35.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:35.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:35.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:35.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:35.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:35.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:35.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:35.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:35.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:36.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:36.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:36.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:36.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:36.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:36.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:36.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:36.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:36.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:37.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:37.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:37.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:37.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:37.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:37.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:37.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:38.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:38.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:39.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:39.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:39.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:39.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:39.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:39.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:39.138 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:39.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:39.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:40.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:40.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:40.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:40.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:40.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:40.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:40.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:40.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:40.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:41.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:41.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:41.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:41.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:41.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:41.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:41.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:41.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:41.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:42.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:42.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:42.596 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:42.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:42.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:42.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:42.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:42.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:42.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:43.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:43.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:43.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:43.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:43.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:43.748 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:43.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:43.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:43.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:44.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:44.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:44.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:44.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:44.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:44.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:44.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:45.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:45.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:46.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:46.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:46.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:46.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:46.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:46.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:46.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:46.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:46.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:47.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:47.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:47.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:47.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:47.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:47.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:47.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:47.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:47.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:48.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:48.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:48.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:48.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:48.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:48.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:48.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:48.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:48.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:49.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:49.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:49.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:49.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:49.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:49.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:49.501 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:49.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:49.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:50.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:50.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:50.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:50.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:50.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:50.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:50.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:50.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:50.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:51.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:51.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:51.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:51.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:51.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:51.797 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:51.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:51.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:51.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:52.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:52.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:52.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:52.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:52.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:52.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:52.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:53.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:53.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:54.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:54.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:54.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:54.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:54.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:54.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:54.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:54.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:54.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:55.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:55.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:55.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:55.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:55.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:55.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:55.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:55.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:55.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:56.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:56.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:56.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:56.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:56.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:56.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:56.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:56.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:56.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:57.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:57.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:57.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:57.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:57.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:57.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:57.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:58.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:58.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:58.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:58.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:58.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:58.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:58.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:58.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:58.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:35:59.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:35:59.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:35:59.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:35:59.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:35:59.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:35:59.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:35:59.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:35:59.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:35:59.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:00.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:00.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:00.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:00.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:00.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:00.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:00.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:01.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:01.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:02.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:02.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:02.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:02.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:02.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:02.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:02.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:02.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:02.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:03.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:03.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:03.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:03.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:03.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:03.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:03.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:03.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:03.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:04.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:04.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:04.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:04.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:04.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:04.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:04.445 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:04.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:04.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:05.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:05.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:05.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:05.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:05.594 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:05.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:05.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:05.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:05.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:06.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:06.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:06.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:06.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:06.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:06.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:06.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:06.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:06.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:07.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:07.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:07.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:07.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:07.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:07.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:07.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:08.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:08.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:09.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:09.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:09.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:09.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:09.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:09.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:09.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:09.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:09.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:10.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:10.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:10.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:10.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:10.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:10.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:10.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:10.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:10.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:11.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:11.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:11.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:11.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:11.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:11.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:11.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:11.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:11.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:12.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:12.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:12.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:12.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:12.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:12.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:12.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:12.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:12.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:13.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:13.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:13.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:13.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:13.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:13.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:13.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:13.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:13.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:14.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:14.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:14.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:14.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:14.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:14.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:14.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:14.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:14.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:15.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:15.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:15.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:15.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:15.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:15.949 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:15.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:16.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:16.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:17.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:17.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:17.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:17.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:17.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:17.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:17.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:17.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:18.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:18.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:18.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:18.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:18.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:18.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:18.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:18.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:18.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:19.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:19.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:19.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:19.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:19.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:19.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:19.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:19.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:19.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:20.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:20.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:20.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:20.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:20.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:20.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:20.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:20.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:20.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:21.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:21.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:21.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:21.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:21.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:21.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:21.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:21.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:21.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:22.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:22.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:22.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:22.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:22.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:22.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:22.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:22.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:22.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:23.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:23.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:23.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:23.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:23.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:23.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:23.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:24.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:24.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:25.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:25.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:25.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:25.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:25.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:25.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:25.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:25.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:25.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:26.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:26.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:26.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:26.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:26.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:26.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:26.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:26.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:26.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:27.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:27.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:27.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:27.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:27.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:27.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:27.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:27.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:27.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:28.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:28.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:28.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:28.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:28.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:28.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:28.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:28.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:28.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:29.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:29.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:29.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:29.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:29.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:29.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:29.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:29.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T06:26:57.383367+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:29.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:36:30.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:36:30.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:36:30.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:36:30.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:36:30.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:36:30.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:36:30.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:36:31.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:36:29.328474+0000 '>' 2026-02-03T06:26:57.383367+0000 2026-02-17T06:36:31.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:36:31.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:405: _scrub_abort: perf_counters td/osd-scrub-test 3 2026-02-17T06:36:31.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:36:31.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T06:36:31.053 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T06:36:31.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T06:36:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:36:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:36:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.118 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.119 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:36:31.120 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.121 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:36:31.122 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.123 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:36:31.124 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.125 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.126 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:36:31.127 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:36:31.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:36:31.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:36:31.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:36:31.189 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.190 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.191 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.192 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.193 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 2, 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 2, 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 1, 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.194 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 335.271723993, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 335.271723993 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 1, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 4.99993101, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 4.99993101 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 67, 2026-02-17T06:36:31.195 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 2, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 2, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.196 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:36:31.197 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.198 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.199 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.200 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.201 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.202 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.203 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.203 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.203 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.203 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.204 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.205 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.205 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.205 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:36:31.205 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:36:31.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:36:31.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T06:36:31.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.259 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.260 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.261 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.262 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.263 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.264 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.265 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:36:31.266 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.267 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.268 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.269 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:36:31.270 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:36:31.271 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:36:31.272 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:36:31.272 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:36:31.272 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:36:31.272 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:36:31.273 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:36:31.274 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.274 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:36:31.274 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:36:31.274 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:36:31.274 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_deep_scrub_abort ------------------ 2026-02-17T06:36:31.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_deep_scrub_abort ------------------' 2026-02-17T06:36:31.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:36:31.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:36:31.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:36:31.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:36:31.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:36:31.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:36:31.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:36:31.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:36:31.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:36:31.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:36:31.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:36:31.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:36:31.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:36:31.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:36:31.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:36:31.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:36:31.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:36:31.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:36:31.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:36:31.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:36:31.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:36:31.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:36:31.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:36:31.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:36:31.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.644 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:36:31.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:36:31.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:36:31.645 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_deep_scrub_abort ------------------ 2026-02-17T06:36:31.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_deep_scrub_abort ------------------' 2026-02-17T06:36:31.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:36:31.646 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_dump_scrub_schedule ------------------- 2026-02-17T06:36:31.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_dump_scrub_schedule -------------------' 2026-02-17T06:36:31.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:36:31.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:36:31.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:36:31.649 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:36:31.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:36:31.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:36:31.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:36:31.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:36:31.651 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:36:31.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:36:31.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:36:31.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:36:31.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:36:31.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:36:31.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:36:31.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:36:31.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:36:31.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:36:31.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:36:31.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:36:31.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:36:31.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:36:31.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:36:31.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:36:31.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:36:31.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:36:31.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:36:31.662 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_dump_scrub_schedule ----------------------- 2026-02-17T06:36:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_dump_scrub_schedule -----------------------' 2026-02-17T06:36:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_dump_scrub_schedule td/osd-scrub-test 2026-02-17T06:36:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:530: TEST_dump_scrub_schedule: local dir=td/osd-scrub-test 2026-02-17T06:36:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:531: TEST_dump_scrub_schedule: local poolname=test 2026-02-17T06:36:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:532: TEST_dump_scrub_schedule: local OSDS=3 2026-02-17T06:36:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:533: TEST_dump_scrub_schedule: local objects=90 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:535: TEST_dump_scrub_schedule: TESTDATA=testdata.507902 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:537: TEST_dump_scrub_schedule: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:36:31.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T06:36:31.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:36:31.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:36:31.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:36:31.683 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:36:31.683 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.683 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:31.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T06:36:31.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:36:31.714 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:36:31.715 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:36:31.716 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.716 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.716 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:36:31.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:36:31.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:36:31.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:36:31.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:36:31.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:36:31.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:36:31.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:36:31.772 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:36:31.772 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:36:31.772 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:36:31.773 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:36:31.773 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.773 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.773 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:36:31.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:36:31.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:36:31.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:538: TEST_dump_scrub_schedule: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:36:31.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:36:31.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:36:31.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:36:31.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:36:31.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:36:31.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:36:31.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:36:31.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:36:31.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:36:31.920 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:36:31.920 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.920 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:31.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:36:31.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:36:31.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:549: TEST_dump_scrub_schedule: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T06:36:31.939 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: expr 3 - 1 2026-02-17T06:36:31.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: seq 0 2 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:553: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 0 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:36:31.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:36:31.945 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:36:31.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:36:31.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:36:31.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:36:31.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T06:36:31.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:36:31.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:36:31.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=318984ea-ca1e-45ad-9eac-8dab06d717df 2026-02-17T06:36:31.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 318984ea-ca1e-45ad-9eac-8dab06d717df' 2026-02-17T06:36:31.948 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 318984ea-ca1e-45ad-9eac-8dab06d717df 2026-02-17T06:36:31.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:36:31.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBvDJRpeG6NOBAAVQy4vtRAEt/8JR3ULM46aA== 2026-02-17T06:36:31.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBvDJRpeG6NOBAAVQy4vtRAEt/8JR3ULM46aA=="}' 2026-02-17T06:36:31.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 318984ea-ca1e-45ad-9eac-8dab06d717df -i td/osd-scrub-test/0/new.json 2026-02-17T06:36:32.043 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:36:32.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:36:32.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 --mkfs --key AQBvDJRpeG6NOBAAVQy4vtRAEt/8JR3ULM46aA== --osd-uuid 318984ea-ca1e-45ad-9eac-8dab06d717df 2026-02-17T06:36:32.069 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:32.058+0000 7f6d7d7dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:32.070 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:32.062+0000 7f6d7d7dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:32.071 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:32.062+0000 7f6d7d7dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:32.071 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:32.062+0000 7f6d7d7dd8c0 -1 bdev(0x55b91bcc5c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:36:32.072 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:32.062+0000 7f6d7d7dd8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:36:34.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:36:34.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:36:34.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:36:34.860 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:36:34.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:36:35.049 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:36:35.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:36:35.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T06:36:35.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:36:35.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:36:35.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:36:35.076 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:35.066+0000 7fadd26518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:35.078 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:35.070+0000 7fadd26518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:35.079 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:35.070+0000 7fadd26518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:35.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:36:35.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:36:35.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:36:35.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:36:35.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:36:35.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:35.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:36:35.205 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:36:35.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:35.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:36:35.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:36.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:36.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:36.355 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:36:36.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:36:36.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:36.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:36:36.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:36.591 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:36.582+0000 7fadd26518c0 -1 Falling back to public interface 2026-02-17T06:36:37.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:37.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:37.507 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:36:37.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:36:37.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:37.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:36:37.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:38.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:38.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:38.661 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:36:38.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:36:38.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:38.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:36:38.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:39.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:39.654+0000 7fadd26518c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:36:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:39.810 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:36:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:36:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:36:39.957 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1818503826,v1:127.0.0.1:6803/1818503826] [v2:127.0.0.1:6804/1818503826,v1:127.0.0.1:6805/1818503826] exists,up 318984ea-ca1e-45ad-9eac-8dab06d717df 2026-02-17T06:36:39.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:36:39.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:36:39.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:36:39.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:36:39.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:553: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 1 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:36:39.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:39.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T06:36:39.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:36:39.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:36:39.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=09ec841b-5cec-4a30-a6ba-ecc33e515446 2026-02-17T06:36:39.966 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 09ec841b-5cec-4a30-a6ba-ecc33e515446 2026-02-17T06:36:39.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 09ec841b-5cec-4a30-a6ba-ecc33e515446' 2026-02-17T06:36:39.967 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:36:39.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB3DJRp56qbORAAxeIBnzjNaoJeX4aSeI579w== 2026-02-17T06:36:39.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB3DJRp56qbORAAxeIBnzjNaoJeX4aSeI579w=="}' 2026-02-17T06:36:39.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 09ec841b-5cec-4a30-a6ba-ecc33e515446 -i td/osd-scrub-test/1/new.json 2026-02-17T06:36:40.115 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:36:40.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:36:40.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 --mkfs --key AQB3DJRp56qbORAAxeIBnzjNaoJeX4aSeI579w== --osd-uuid 09ec841b-5cec-4a30-a6ba-ecc33e515446 2026-02-17T06:36:40.143 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:40.134+0000 7f0c9051a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:40.144 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:40.134+0000 7f0c9051a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:40.146 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:40.138+0000 7f0c9051a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:40.146 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:40.138+0000 7f0c9051a8c0 -1 bdev(0x556e0d601c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:36:40.147 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:40.138+0000 7f0c9051a8c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:36:42.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:36:42.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:36:42.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:36:42.411 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:36:42.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:36:42.602 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:36:42.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:36:42.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T06:36:42.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:36:42.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:36:42.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:36:42.626 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:42.618+0000 7fb3a85af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:42.628 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:42.618+0000 7fb3a85af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:42.629 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:42.618+0000 7fb3a85af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:42.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:36:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:36:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:36:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:36:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:36:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:42.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:36:42.759 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:36:42.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:42.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:36:42.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:43.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:43.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:43.909 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:36:43.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:36:43.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:43.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:36:44.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:44.099 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:44.090+0000 7fb3a85af8c0 -1 Falling back to public interface 2026-02-17T06:36:45.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:45.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:45.060 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:36:45.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:36:45.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:45.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:36:45.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:46.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:46.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:46.205 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:36:46.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:36:46.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:46.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:36:46.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:47.121 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:47.110+0000 7fb3a85af8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:36:47.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:47.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:47.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:36:47.359 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:36:47.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:47.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:36:47.509 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1937562873,v1:127.0.0.1:6811/1937562873] [v2:127.0.0.1:6812/1937562873,v1:127.0.0.1:6813/1937562873] exists,up 09ec841b-5cec-4a30-a6ba-ecc33e515446 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:553: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 2 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:36:47.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:36:47.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:36:47.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:36:47.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:36:47.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=dc16bf64-9aad-40ef-8623-81623da1be6e 2026-02-17T06:36:47.517 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 dc16bf64-9aad-40ef-8623-81623da1be6e 2026-02-17T06:36:47.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 dc16bf64-9aad-40ef-8623-81623da1be6e' 2026-02-17T06:36:47.518 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:36:47.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB/DJRp3kDyHhAAOLLPDJkiRnNjzUAaDUWHxQ== 2026-02-17T06:36:47.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB/DJRp3kDyHhAAOLLPDJkiRnNjzUAaDUWHxQ=="}' 2026-02-17T06:36:47.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new dc16bf64-9aad-40ef-8623-81623da1be6e -i td/osd-scrub-test/2/new.json 2026-02-17T06:36:47.677 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:36:47.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T06:36:47.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 --mkfs --key AQB/DJRp3kDyHhAAOLLPDJkiRnNjzUAaDUWHxQ== --osd-uuid dc16bf64-9aad-40ef-8623-81623da1be6e 2026-02-17T06:36:47.704 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:47.694+0000 7f507f82d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:47.706 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:47.698+0000 7f507f82d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:47.707 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:47.698+0000 7f507f82d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:47.707 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:47.698+0000 7f507f82d8c0 -1 bdev(0x5616f117dc00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:36:47.707 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:47.698+0000 7f507f82d8c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:36:49.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T06:36:49.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:36:49.973 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:36:49.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:36:49.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:36:50.160 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:36:50.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:36:50.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T06:36:50.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:36:50.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:36:50.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:36:50.186 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:50.178+0000 7f2e008998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:50.188 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:50.178+0000 7f2e008998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:50.188 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:50.178+0000 7f2e008998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:36:50.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:36:50.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:36:50.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:36:50.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:36:50.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:36:50.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:50.315 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:36:50.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:36:50.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:50.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:36:50.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:51.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:51.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:51.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:36:51.463 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:36:51.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:51.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:36:51.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:51.703 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:51.694+0000 7f2e008998c0 -1 Falling back to public interface 2026-02-17T06:36:52.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:52.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:52.612 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:36:52.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:36:52.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:52.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:36:52.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:53.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:53.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:53.760 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:36:53.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:36:53.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:53.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:36:53.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:54.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:54.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:54.909 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:36:54.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:36:54.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:54.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:36:54.933 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:36:54.922+0000 7f2e008998c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:36:55.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:36:56.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:36:56.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:36:56.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:36:56.058 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:36:56.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:36:56.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:36:56.210 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3528290257,v1:127.0.0.1:6819/3528290257] [v2:127.0.0.1:6820/3528290257,v1:127.0.0.1:6821/3528290257] exists,up dc16bf64-9aad-40ef-8623-81623da1be6e 2026-02-17T06:36:56.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:36:56.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:36:56.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:36:56.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:557: TEST_dump_scrub_schedule: create_pool test 1 1 2026-02-17T06:36:56.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:36:56.409 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:36:56.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:36:57.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:558: TEST_dump_scrub_schedule: wait_for_clean 2026-02-17T06:36:57.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:36:57.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:36:57.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:36:57.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:36:57.422 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:36:57.422 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:36:57.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:36:57.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:36:57.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:36:57.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:36:57.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:36:57.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:36:57.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:36:57.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:36:57.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:36:57.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:36:57.685 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:36:57.685 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:36:57.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:36:57.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:36:57.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:36:57.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T06:36:57.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T06:36:57.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T06:36:57.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:36:57.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:36:57.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T06:36:57.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T06:36:57.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705675' 2026-02-17T06:36:57.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:36:57.827 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:36:57.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-17T06:36:57.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-17T06:36:57.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705675 2-55834574851' 2026-02-17T06:36:57.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:36:57.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T06:36:57.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:36:57.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:36:57.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T06:36:57.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:36:57.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T06:36:57.903 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836498 2026-02-17T06:36:57.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T06:36:57.903 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:36:58.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836498 2026-02-17T06:36:58.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:36:59.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:36:59.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:36:59.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836498 2026-02-17T06:36:59.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:36:59.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T06:36:59.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:36:59.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:36:59.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T06:36:59.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:36:59.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T06:36:59.210 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705675 2026-02-17T06:36:59.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T06:36:59.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:36:59.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T06:36:59.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:36:59.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-17T06:36:59.359 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:36:59.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:36:59.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-17T06:36:59.361 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:36:59.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-17T06:36:59.362 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574851 2026-02-17T06:36:59.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-17T06:36:59.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:36:59.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574851 2026-02-17T06:36:59.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:36:59.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:36:59.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:36:59.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:36:59.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:36:59.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:36:59.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:36:59.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:36:59.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:36:59.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:36:59.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:36:59.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:36:59.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:36:59.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:36:59.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:37:00.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:37:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T06:37:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T06:37:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T06:37:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:37:00.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T06:37:00.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:37:00.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:37:00.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:37:00.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:37:00.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:37:00.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:37:00.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:37:00.132 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:37:00.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:37:00.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:37:00.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:37:00.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:37:00.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:37:00.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:37:00.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:37:00.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: ceph osd dump 2026-02-17T06:37:00.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: grep '^pool.*['\'']test['\'']' 2026-02-17T06:37:00.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: awk '{ print $2 }' 2026-02-17T06:37:00.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: poolid=1 2026-02-17T06:37:00.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:561: TEST_dump_scrub_schedule: dd if=/dev/urandom of=testdata.507902 bs=1032 count=1 2026-02-17T06:37:00.605 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:37:00.605 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:37:00.605 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.7658e-05 s, 17.9 MB/s 2026-02-17T06:37:00.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: seq 1 90 2026-02-17T06:37:00.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj1 testdata.507902 2026-02-17T06:37:00.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj2 testdata.507902 2026-02-17T06:37:00.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj3 testdata.507902 2026-02-17T06:37:00.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj4 testdata.507902 2026-02-17T06:37:00.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj5 testdata.507902 2026-02-17T06:37:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj6 testdata.507902 2026-02-17T06:37:00.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj7 testdata.507902 2026-02-17T06:37:00.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj8 testdata.507902 2026-02-17T06:37:00.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj9 testdata.507902 2026-02-17T06:37:00.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj10 testdata.507902 2026-02-17T06:37:00.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj11 testdata.507902 2026-02-17T06:37:00.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj12 testdata.507902 2026-02-17T06:37:00.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj13 testdata.507902 2026-02-17T06:37:00.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj14 testdata.507902 2026-02-17T06:37:00.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj15 testdata.507902 2026-02-17T06:37:00.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj16 testdata.507902 2026-02-17T06:37:00.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:00.982 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj17 testdata.507902 2026-02-17T06:37:01.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj18 testdata.507902 2026-02-17T06:37:01.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj19 testdata.507902 2026-02-17T06:37:01.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj20 testdata.507902 2026-02-17T06:37:01.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj21 testdata.507902 2026-02-17T06:37:01.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj22 testdata.507902 2026-02-17T06:37:01.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj23 testdata.507902 2026-02-17T06:37:01.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj24 testdata.507902 2026-02-17T06:37:01.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj25 testdata.507902 2026-02-17T06:37:01.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj26 testdata.507902 2026-02-17T06:37:01.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj27 testdata.507902 2026-02-17T06:37:01.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj28 testdata.507902 2026-02-17T06:37:01.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj29 testdata.507902 2026-02-17T06:37:01.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj30 testdata.507902 2026-02-17T06:37:01.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj31 testdata.507902 2026-02-17T06:37:01.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj32 testdata.507902 2026-02-17T06:37:01.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj33 testdata.507902 2026-02-17T06:37:01.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj34 testdata.507902 2026-02-17T06:37:01.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj35 testdata.507902 2026-02-17T06:37:01.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj36 testdata.507902 2026-02-17T06:37:01.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj37 testdata.507902 2026-02-17T06:37:01.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj38 testdata.507902 2026-02-17T06:37:01.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj39 testdata.507902 2026-02-17T06:37:01.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj40 testdata.507902 2026-02-17T06:37:01.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj41 testdata.507902 2026-02-17T06:37:01.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj42 testdata.507902 2026-02-17T06:37:01.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj43 testdata.507902 2026-02-17T06:37:01.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj44 testdata.507902 2026-02-17T06:37:01.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj45 testdata.507902 2026-02-17T06:37:01.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj46 testdata.507902 2026-02-17T06:37:01.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj47 testdata.507902 2026-02-17T06:37:01.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj48 testdata.507902 2026-02-17T06:37:01.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj49 testdata.507902 2026-02-17T06:37:01.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj50 testdata.507902 2026-02-17T06:37:01.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj51 testdata.507902 2026-02-17T06:37:01.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj52 testdata.507902 2026-02-17T06:37:01.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj53 testdata.507902 2026-02-17T06:37:01.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj54 testdata.507902 2026-02-17T06:37:01.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj55 testdata.507902 2026-02-17T06:37:01.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj56 testdata.507902 2026-02-17T06:37:01.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj57 testdata.507902 2026-02-17T06:37:01.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj58 testdata.507902 2026-02-17T06:37:01.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj59 testdata.507902 2026-02-17T06:37:01.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:01.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj60 testdata.507902 2026-02-17T06:37:02.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj61 testdata.507902 2026-02-17T06:37:02.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj62 testdata.507902 2026-02-17T06:37:02.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj63 testdata.507902 2026-02-17T06:37:02.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj64 testdata.507902 2026-02-17T06:37:02.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj65 testdata.507902 2026-02-17T06:37:02.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj66 testdata.507902 2026-02-17T06:37:02.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj67 testdata.507902 2026-02-17T06:37:02.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj68 testdata.507902 2026-02-17T06:37:02.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj69 testdata.507902 2026-02-17T06:37:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj70 testdata.507902 2026-02-17T06:37:02.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj71 testdata.507902 2026-02-17T06:37:02.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj72 testdata.507902 2026-02-17T06:37:02.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj73 testdata.507902 2026-02-17T06:37:02.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj74 testdata.507902 2026-02-17T06:37:02.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj75 testdata.507902 2026-02-17T06:37:02.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj76 testdata.507902 2026-02-17T06:37:02.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj77 testdata.507902 2026-02-17T06:37:02.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj78 testdata.507902 2026-02-17T06:37:02.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj79 testdata.507902 2026-02-17T06:37:02.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj80 testdata.507902 2026-02-17T06:37:02.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj81 testdata.507902 2026-02-17T06:37:02.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj82 testdata.507902 2026-02-17T06:37:02.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj83 testdata.507902 2026-02-17T06:37:02.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj84 testdata.507902 2026-02-17T06:37:02.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj85 testdata.507902 2026-02-17T06:37:02.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj86 testdata.507902 2026-02-17T06:37:02.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj87 testdata.507902 2026-02-17T06:37:02.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj88 testdata.507902 2026-02-17T06:37:02.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj89 testdata.507902 2026-02-17T06:37:02.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T06:37:02.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj90 testdata.507902 2026-02-17T06:37:02.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:566: TEST_dump_scrub_schedule: rm -f testdata.507902 2026-02-17T06:37:02.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:568: TEST_dump_scrub_schedule: local pgid=1.0 2026-02-17T06:37:02.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:570: TEST_dump_scrub_schedule: date +%Y-%m-%dT%H:%M:%S.%N%:z 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:570: TEST_dump_scrub_schedule: local now_is=2026-02-17T06:37:02.708963303+00:00 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:578: TEST_dump_scrub_schedule: expct_starting=(['query_active']='false' ['query_is_future']='true' ['query_schedule']='scrub scheduled') 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:578: TEST_dump_scrub_schedule: declare -A expct_starting 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:579: TEST_dump_scrub_schedule: declare -A sched_data 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:580: TEST_dump_scrub_schedule: extract_published_sch 1.0 2026-02-17T06:37:02.708963303+00:00 2019-10-12T20:32:43.645168+0000 sched_data 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.0 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sched_data 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-17T06:37:02.708963303+00:00 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2019-10-12T20:32:43.645168+0000 2026-02-17T06:37:02.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-17T06:37:02.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-17T06:37:02.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-17T06:37:02.852 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:02.861 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:02.861 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 0, 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-18T06:36:56.403369+0000", 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 17, 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 44 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:02.862 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:02.932 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-18T06:36:56.403369+0000", 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-18T06:36:56.403 (2026-02-18T06:36:56.403)" 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:02.933 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:03.013 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:03.013 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=17 2026-02-17T06:37:03.013 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=106 2026-02-17T06:37:03.013 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:03.013 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:37:03.013 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-18T06:36:56.403' 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-18T06:36:56.403' 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=0 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-17T06:36:56.403369+0000' 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='0x0' 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=true 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=0 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:03.014 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-18T06:36:56.403369+0000' 2026-02-17T06:37:03.015 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:03.015 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:03.015 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=17 2026-02-17T06:37:03.015 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=44 2026-02-17T06:37:03.015 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:03.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:581: TEST_dump_scrub_schedule: schedule_against_expected sched_data expct_starting initial 2026-02-17T06:37:03.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:178: schedule_against_expected: local -n dict=sched_data 2026-02-17T06:37:03.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:179: schedule_against_expected: local -n ep=expct_starting 2026-02-17T06:37:03.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:180: schedule_against_expected: local extr_dbg=1 2026-02-17T06:37:03.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:183: schedule_against_expected: local saved_echo_flag=x 2026-02-17T06:37:03.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:184: schedule_against_expected: set +x 2026-02-17T06:37:03.026 INFO:tasks.workunit.client.0.trial074.stdout:-- - comparing: 2026-02-17T06:37:03.027 INFO:tasks.workunit.client.0.trial074.stdout:key is query_schedule expected: scrub scheduled in actual: scrub scheduled 2026-02-17T06:37:03.027 INFO:tasks.workunit.client.0.trial074.stdout:key is query_is_future expected: true in actual: true 2026-02-17T06:37:03.027 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active expected: false in actual: false 2026-02-17T06:37:03.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:203: schedule_against_expected: return 0 2026-02-17T06:37:03.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:582: TEST_dump_scrub_schedule: (( 0 == 0 )) 2026-02-17T06:37:03.028 INFO:tasks.workunit.client.0.trial074.stdout:last-scrub --- 0x0 2026-02-17T06:37:03.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:583: TEST_dump_scrub_schedule: echo 'last-scrub --- ' 0x0 2026-02-17T06:37:03.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:589: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-17T06:36:56.403369+0000 2026-02-17T06:37:03.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:590: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_scrub_sleep 0 2026-02-17T06:37:03.088 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:37:03.088 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T06:37:03.089 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:03.094 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:37:03.095 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T06:37:03.095 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:03.100 INFO:tasks.workunit.client.0.trial074.stdout:osd.2: { 2026-02-17T06:37:03.100 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T06:37:03.101 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:03.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:591: TEST_dump_scrub_schedule: ceph tell 1.0 deep-scrub 2026-02-17T06:37:03.167 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:37:03.167 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:37:03.167 INFO:tasks.workunit.client.0.trial074.stdout: "must": true, 2026-02-17T06:37:03.167 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "0.000000" 2026-02-17T06:37:03.167 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:03.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:596: TEST_dump_scrub_schedule: sleep 5 2026-02-17T06:37:08.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:597: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T06:37:08.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:598: TEST_dump_scrub_schedule: expct_qry_duration=(['query_last_duration']='0' ['query_last_duration_neg']='not0') 2026-02-17T06:37:08.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:598: TEST_dump_scrub_schedule: declare -A expct_qry_duration 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:599: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T06:36:56.403369+0000 expct_qry_duration 'WaitingAfterScrub ' sched_data 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T06:36:56.403369+0000 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_qry_duration 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T06:37:08.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T06:37:08.177 INFO:tasks.workunit.client.0.trial074.stdout:waiting for any condition (WaitingAfterScrub ): pg:1.0 dt:2026-02-17T06:36:56.403369+0000 (10 retries) 2026-02-17T06:37:08.820 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:08.831 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:08.831 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:08.831 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-18T06:37:04.167917+0000", 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 17, 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 123 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:08.832 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:08.903 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:08.903 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:08.903 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:08.903 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:08.903 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:08.903 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:08.903 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-18T06:37:04.167917+0000", 2026-02-17T06:37:08.904 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-18T06:37:04.167 (2026-02-18T06:37:04.167)" 2026-02-17T06:37:08.904 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:08.904 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=17 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=124 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-18T06:37:04.167' 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-18T06:37:04.167' 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-17T06:37:04.167917+0000' 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=true 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:08.984 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:08.985 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-18T06:37:04.167917+0000' 2026-02-17T06:37:08.985 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:08.985 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:08.985 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=17 2026-02-17T06:37:08.985 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=123 2026-02-17T06:37:08.985 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 1 ~ false / 124 / 123 / true / 2026-02-17T06:37:04.167917+0000 / scrub scheduled %%% query_last_duration_neg query_last_duration 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stdout:key is query_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stdout:WaitingAfterScrub - 'query_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:601: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:602: TEST_dump_scrub_schedule: expct_dmp_duration=(['dmp_last_duration']='0' ['dmp_last_duration_neg']='not0') 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:602: TEST_dump_scrub_schedule: declare -A expct_dmp_duration 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:603: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T06:36:56.403369+0000 expct_dmp_duration 'WaitingAfterScrub_dmp ' sched_data 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T06:37:08.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T06:37:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T06:36:56.403369+0000 2026-02-17T06:37:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_dmp_duration 2026-02-17T06:37:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T06:37:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T06:37:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T06:37:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T06:37:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T06:37:08.997 INFO:tasks.workunit.client.0.trial074.stdout:waiting for any condition (WaitingAfterScrub_dmp ): pg:1.0 dt:2026-02-17T06:36:56.403369+0000 (10 retries) 2026-02-17T06:37:09.642 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-18T06:37:04.167917+0000", 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:09.652 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 17, 2026-02-17T06:37:09.653 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 123 2026-02-17T06:37:09.653 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:09.653 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:09.720 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:09.720 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:09.720 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:09.720 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:09.720 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:09.720 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:09.721 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-18T06:37:04.167917+0000", 2026-02-17T06:37:09.721 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-18T06:37:04.167 (2026-02-18T06:37:04.167)" 2026-02-17T06:37:09.721 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:09.721 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:09.801 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:09.801 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=17 2026-02-17T06:37:09.801 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=125 2026-02-17T06:37:09.801 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-18T06:37:04.167' 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-18T06:37:04.167' 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-17T06:37:04.167917+0000' 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=true 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:09.802 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-18T06:37:04.167917+0000' 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=17 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=123 2026-02-17T06:37:09.803 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:09.813 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 1 ~ false / 125 / 123 / true / 2026-02-17T06:37:04.167917+0000 / scrub scheduled %%% dmp_last_duration dmp_last_duration_neg 2026-02-17T06:37:09.813 INFO:tasks.workunit.client.0.trial074.stdout:key is dmp_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-17T06:37:09.813 INFO:tasks.workunit.client.0.trial074.stdout:WaitingAfterScrub_dmp - 'dmp_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-17T06:37:09.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T06:37:09.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:610: TEST_dump_scrub_schedule: ceph osd set noscrub 2026-02-17T06:37:10.009 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:37:10.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:611: TEST_dump_scrub_schedule: sleep 2 2026-02-17T06:37:12.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:612: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-17T06:37:12.085 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:37:12.085 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) " 2026-02-17T06:37:12.085 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:12.091 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:37:12.091 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) " 2026-02-17T06:37:12.091 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:12.097 INFO:tasks.workunit.client.0.trial074.stdout:osd.2: { 2026-02-17T06:37:12.098 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) " 2026-02-17T06:37:12.098 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:12.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:613: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_scrub_sleep 2.0 2026-02-17T06:37:12.179 INFO:tasks.workunit.client.0.trial074.stdout:osd.0: { 2026-02-17T06:37:12.180 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T06:37:12.180 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:12.186 INFO:tasks.workunit.client.0.trial074.stdout:osd.1: { 2026-02-17T06:37:12.186 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T06:37:12.186 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:12.192 INFO:tasks.workunit.client.0.trial074.stdout:osd.2: { 2026-02-17T06:37:12.192 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T06:37:12.192 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:12.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:614: TEST_dump_scrub_schedule: sleep 8 2026-02-17T06:37:20.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:615: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-17T06:37:04.167917+0000 2026-02-17T06:37:20.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:617: TEST_dump_scrub_schedule: ceph tell 1.0 schedule-scrub 2026-02-17T06:37:20.264 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:37:20.264 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:37:20.264 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:37:20.264 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:35:40.259117+0000" 2026-02-17T06:37:20.264 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:20.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:618: TEST_dump_scrub_schedule: sleep 1 2026-02-17T06:37:21.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:619: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:621: TEST_dump_scrub_schedule: expct_scrub_peri_sched=(['query_is_future']='false') 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:621: TEST_dump_scrub_schedule: declare -A expct_scrub_peri_sched 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:622: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T06:37:04.167917+0000 expct_scrub_peri_sched waitingBeingScheduled sched_data 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T06:37:04.167917+0000 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_scrub_peri_sched 2026-02-17T06:37:21.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T06:37:21.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T06:37:21.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T06:37:21.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T06:37:21.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T06:37:21.275 INFO:tasks.workunit.client.0.trial074.stdout:waiting for any condition (waitingBeingScheduled): pg:1.0 dt:2026-02-17T06:37:04.167917+0000 (10 retries) 2026-02-17T06:37:21.920 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-18T06:37:04.167917+0000", 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:21.929 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:21.930 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 18, 2026-02-17T06:37:21.930 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 132 2026-02-17T06:37:21.930 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:21.930 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:22.000 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-17T06:37:25.324 (2026-02-11T06:35:40.259)" 2026-02-17T06:37:22.001 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:22.002 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:22.081 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=18 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=136 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-17T06:37:25.324' 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-11T06:35:40.259' 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:22.082 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:22.083 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:22.083 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-18T06:37:04.167917+0000' 2026-02-17T06:37:22.083 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:22.083 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:22.083 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=18 2026-02-17T06:37:22.083 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=132 2026-02-17T06:37:22.083 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:22.093 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 1 ~ false / 136 / 132 / false / 2026-02-10T06:35:40.259117+0000 / scrub scheduled %%% query_is_future 2026-02-17T06:37:22.094 INFO:tasks.workunit.client.0.trial074.stdout:key is query_is_future: negation:0 # expected: false # in actual: false 2026-02-17T06:37:22.094 INFO:tasks.workunit.client.0.trial074.stdout:waitingBeingScheduled - 'query_is_future' actual value (false) matches expected (false) (negation: 0) 2026-02-17T06:37:22.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T06:37:22.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:634: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-10T06:35:40.259117+0000 2026-02-17T06:37:22.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:635: TEST_dump_scrub_schedule: ceph osd unset noscrub 2026-02-17T06:37:22.332 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is unset 2026-02-17T06:37:22.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:637: TEST_dump_scrub_schedule: cond_active=(['query_active']='true') 2026-02-17T06:37:22.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:637: TEST_dump_scrub_schedule: declare -A cond_active 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:638: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:639: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-10T06:35:40.259117+0000 cond_active 'WaitingActive ' sched_data 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-10T06:35:40.259117+0000 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=cond_active 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T06:37:22.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T06:37:22.345 INFO:tasks.workunit.client.0.trial074.stdout:waiting for any condition (WaitingActive ): pg:1.0 dt:2026-02-10T06:35:40.259117+0000 (10 retries) 2026-02-17T06:37:22.989 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:22.999 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 18, 2026-02-17T06:37:23.000 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 135 2026-02-17T06:37:23.000 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:23.000 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:23.071 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:23.071 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:23.071 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:23.071 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:23.072 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:23.072 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:23.072 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:23.072 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-17T06:37:25.324 (2026-02-11T06:35:40.259)" 2026-02-17T06:37:23.072 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:23.072 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:23.153 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=138 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-17T06:37:25.324' 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-11T06:35:40.259' 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:23.154 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:37:25.324052+0000' 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=18 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=135 2026-02-17T06:37:23.155 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:23.165 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 1 ~ false / 138 / 135 / false / 2026-02-10T06:35:40.259117+0000 / scrub scheduled %%% query_active 2026-02-17T06:37:23.166 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T06:37:23.809 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:23.818 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:23.819 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 18, 2026-02-17T06:37:23.819 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 135 2026-02-17T06:37:23.819 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:23.819 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:23.889 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-17T06:37:25.324 (2026-02-11T06:35:40.259)" 2026-02-17T06:37:23.890 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:23.890 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:23.972 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:23.972 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:23.972 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=139 2026-02-17T06:37:23.972 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:23.972 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:37:23.972 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-17T06:37:25.324' 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-11T06:35:40.259' 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:23.973 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:23.974 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:37:25.324052+0000' 2026-02-17T06:37:23.974 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:23.974 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:23.974 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=18 2026-02-17T06:37:23.974 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=135 2026-02-17T06:37:23.974 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:23.982 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 2 ~ false / 139 / 135 / false / 2026-02-10T06:35:40.259117+0000 / scrub scheduled %%% query_active 2026-02-17T06:37:23.983 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T06:37:24.626 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:24.635 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 19, 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 138 2026-02-17T06:37:24.636 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:24.637 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:24.708 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:24.708 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:24.708 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:24.709 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:24.709 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:24.709 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:24.709 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:24.709 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-17T06:37:25.324 (2026-02-11T06:35:40.259)" 2026-02-17T06:37:24.709 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:24.710 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:24.790 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:24.790 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:24.790 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=139 2026-02-17T06:37:24.790 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:24.790 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:37:24.790 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-17T06:37:25.324' 2026-02-17T06:37:24.790 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-11T06:35:40.259' 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:24.791 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:37:25.324052+0000' 2026-02-17T06:37:24.792 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:24.792 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:24.792 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=19 2026-02-17T06:37:24.792 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=138 2026-02-17T06:37:24.792 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:24.801 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 3 ~ false / 139 / 138 / false / 2026-02-10T06:35:40.259117+0000 / scrub scheduled %%% query_active 2026-02-17T06:37:24.801 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T06:37:25.442 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 19, 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 138 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:25.451 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:25.523 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:25.523 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:37:25.523 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:37:25.523 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:37:25.523 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:37:25.524 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:37:25.524 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:25.524 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "queued for scrub" 2026-02-17T06:37:25.524 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:25.524 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:25.605 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:25.605 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:25.605 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=140 2026-02-17T06:37:25.605 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='queued for scrub' 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='0' 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='0' 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=false 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:25.606 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:25.607 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:25.607 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:37:25.324052+0000' 2026-02-17T06:37:25.607 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:25.607 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:25.607 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=19 2026-02-17T06:37:25.607 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=138 2026-02-17T06:37:25.607 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:25.617 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 4 ~ false / 140 / 138 / false / 2026-02-10T06:35:40.259117+0000 / queued for scrub %%% query_active 2026-02-17T06:37:25.617 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T06:37:26.258 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:26.269 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 19, 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 139 2026-02-17T06:37:26.270 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:26.271 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:26.343 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:26.343 INFO:tasks.workunit.client.0.trial074.stdout: "active": true, 2026-02-17T06:37:26.343 INFO:tasks.workunit.client.0.trial074.stdout: "epoch_start": "15", 2026-02-17T06:37:26.343 INFO:tasks.workunit.client.0.trial074.stdout: "start": "1:00000000::::0", 2026-02-17T06:37:26.343 INFO:tasks.workunit.client.0.trial074.stdout: "end": "MIN", 2026-02-17T06:37:26.343 INFO:tasks.workunit.client.0.trial074.stdout: "max_end": "MIN", 2026-02-17T06:37:26.343 INFO:tasks.workunit.client.0.trial074.stdout: "subset_last_update": "0'0", 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "req_scrub": false, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "auto_repair": false, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "check_repair": false, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "deep_scrub_on_error": false, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "priority": 5, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "shallow_errors": 0, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "deep_errors": 0, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "fixed": 0, 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "waiting_on_whom": [], 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrubbing" 2026-02-17T06:37:26.344 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:26.345 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:26.425 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:26.425 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=144 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=true 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrubbing' 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='0' 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='0' 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:26.426 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=false 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:37:25.324052+0000' 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=19 2026-02-17T06:37:26.427 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=139 2026-02-17T06:37:26.428 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:26.437 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 5 ~ true / 144 / 139 / false / 2026-02-10T06:35:40.259117+0000 / scrubbing %%% query_active 2026-02-17T06:37:26.437 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active: negation:0 # expected: true # in actual: true 2026-02-17T06:37:26.437 INFO:tasks.workunit.client.0.trial074.stdout:WaitingActive - 'query_active' actual value (true) matches expected (true) (negation: 0) 2026-02-17T06:37:26.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T06:37:26.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:643: TEST_dump_scrub_schedule: cond_active_dmp=(['dmp_state_has_scrubbing']='true' ['query_active']='false') 2026-02-17T06:37:26.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:643: TEST_dump_scrub_schedule: declare -A cond_active_dmp 2026-02-17T06:37:26.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:644: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:645: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-10T06:35:40.259117+0000 cond_active_dmp 'WaitingActive ' sched_data 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-10T06:35:40.259117+0000 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=cond_active_dmp 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T06:37:26.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T06:37:26.441 INFO:tasks.workunit.client.0.trial074.stdout:waiting for any condition (WaitingActive ): pg:1.0 dt:2026-02-10T06:35:40.259117+0000 (10 retries) 2026-02-17T06:37:27.085 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": false, 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 19, 2026-02-17T06:37:27.095 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 139 2026-02-17T06:37:27.096 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:27.096 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:27.166 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:27.166 INFO:tasks.workunit.client.0.trial074.stdout: "active": true, 2026-02-17T06:37:27.166 INFO:tasks.workunit.client.0.trial074.stdout: "epoch_start": "15", 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "start": "1:00000000::::0", 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "end": "MIN", 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "max_end": "MIN", 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "subset_last_update": "0'0", 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "req_scrub": false, 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "auto_repair": false, 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "check_repair": false, 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "deep_scrub_on_error": false, 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "priority": 5, 2026-02-17T06:37:27.167 INFO:tasks.workunit.client.0.trial074.stdout: "shallow_errors": 0, 2026-02-17T06:37:27.168 INFO:tasks.workunit.client.0.trial074.stdout: "deep_errors": 0, 2026-02-17T06:37:27.168 INFO:tasks.workunit.client.0.trial074.stdout: "fixed": 0, 2026-02-17T06:37:27.168 INFO:tasks.workunit.client.0.trial074.stdout: "waiting_on_whom": [], 2026-02-17T06:37:27.168 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrubbing" 2026-02-17T06:37:27.168 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:27.168 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:27.250 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:27.250 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:27.250 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=144 2026-02-17T06:37:27.250 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=true 2026-02-17T06:37:27.250 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrubbing' 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='0' 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='0' 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=false 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:27.251 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:27.252 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:37:25.324052+0000' 2026-02-17T06:37:27.252 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=false 2026-02-17T06:37:27.252 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:27.252 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=19 2026-02-17T06:37:27.252 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=139 2026-02-17T06:37:27.252 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:27.261 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 1 ~ true / 144 / 139 / false / 2026-02-10T06:35:40.259117+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-17T06:37:27.261 INFO:tasks.workunit.client.0.trial074.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: false 2026-02-17T06:37:27.261 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active: negation:0 # expected: false # in actual: true 2026-02-17T06:37:27.905 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:27.914 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:27.914 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:37:25.324052+0000", 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": false, 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 19, 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 139 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:27.915 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:27.987 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:27.987 INFO:tasks.workunit.client.0.trial074.stdout: "active": true, 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "epoch_start": "15", 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "start": "1:00000000::::0", 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "end": "MIN", 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "max_end": "MIN", 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "subset_last_update": "0'0", 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "req_scrub": false, 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "auto_repair": false, 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "check_repair": false, 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "deep_scrub_on_error": false, 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "priority": 5, 2026-02-17T06:37:27.988 INFO:tasks.workunit.client.0.trial074.stdout: "shallow_errors": 0, 2026-02-17T06:37:27.989 INFO:tasks.workunit.client.0.trial074.stdout: "deep_errors": 0, 2026-02-17T06:37:27.989 INFO:tasks.workunit.client.0.trial074.stdout: "fixed": 0, 2026-02-17T06:37:27.989 INFO:tasks.workunit.client.0.trial074.stdout: "waiting_on_whom": [], 2026-02-17T06:37:27.989 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrubbing" 2026-02-17T06:37:27.989 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:27.989 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:28.069 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:28.069 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:28.069 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=144 2026-02-17T06:37:28.069 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=true 2026-02-17T06:37:28.069 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrubbing' 2026-02-17T06:37:28.069 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='0' 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='0' 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=false 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:37:28.070 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:37:25.324052+0000' 2026-02-17T06:37:28.071 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=false 2026-02-17T06:37:28.071 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:37:28.071 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=19 2026-02-17T06:37:28.071 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=139 2026-02-17T06:37:28.071 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:28.081 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 2 ~ true / 144 / 139 / false / 2026-02-10T06:35:40.259117+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-17T06:37:28.081 INFO:tasks.workunit.client.0.trial074.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: false 2026-02-17T06:37:28.081 INFO:tasks.workunit.client.0.trial074.stdout:key is query_active: negation:0 # expected: false # in actual: true 2026-02-17T06:37:28.726 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean+scrubbing", 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": true, 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 1, 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "scrubbing for 0s", 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "0", 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": false, 2026-02-17T06:37:28.736 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": false, 2026-02-17T06:37:28.737 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 19, 2026-02-17T06:37:28.737 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 143 2026-02-17T06:37:28.737 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:28.737 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:37:28.807 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:37:28.807 INFO:tasks.workunit.client.0.trial074.stdout: "active": true, 2026-02-17T06:37:28.807 INFO:tasks.workunit.client.0.trial074.stdout: "epoch_start": "15", 2026-02-17T06:37:28.807 INFO:tasks.workunit.client.0.trial074.stdout: "start": "1:a0217245:::obj31:0", 2026-02-17T06:37:28.807 INFO:tasks.workunit.client.0.trial074.stdout: "end": "1:a0217245:::obj31:0", 2026-02-17T06:37:28.807 INFO:tasks.workunit.client.0.trial074.stdout: "max_end": "1:a0217245:::obj31:0", 2026-02-17T06:37:28.807 INFO:tasks.workunit.client.0.trial074.stdout: "subset_last_update": "17'90", 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "req_scrub": false, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "auto_repair": false, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "check_repair": false, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "deep_scrub_on_error": false, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "priority": 5, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "shallow_errors": 0, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "deep_errors": 0, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "fixed": 0, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "waiting_on_whom": [], 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrubbing" 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:28.808 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:37:28.893 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:37:28.893 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=19 2026-02-17T06:37:28.893 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=146 2026-02-17T06:37:28.893 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=true 2026-02-17T06:37:28.893 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrubbing' 2026-02-17T06:37:28.893 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='0' 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='0' 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-10T06:35:40.259117+0000' 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='17x90' 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=false 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=null 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean+scrubbing' 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=true 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=1 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='scrubbing for 0s' 2026-02-17T06:37:28.894 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='0' 2026-02-17T06:37:28.895 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=false 2026-02-17T06:37:28.895 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=false 2026-02-17T06:37:28.895 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=19 2026-02-17T06:37:28.895 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=143 2026-02-17T06:37:28.895 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:37:28.905 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 3 ~ true / 146 / 143 / false / 2026-02-10T06:35:40.259117+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-17T06:37:28.905 INFO:tasks.workunit.client.0.trial074.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: true 2026-02-17T06:37:28.905 INFO:tasks.workunit.client.0.trial074.stdout:WaitingActive - 'dmp_state_has_scrubbing' actual value (true) matches expected (true) (negation: 0) 2026-02-17T06:37:28.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T06:37:28.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:646: TEST_dump_scrub_schedule: sleep 4 2026-02-17T06:37:32.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:647: TEST_dump_scrub_schedule: perf_counters td/osd-scrub-test 3 2026-02-17T06:37:32.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:37:32.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T06:37:32.908 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T06:37:32.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T06:37:32.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:37:32.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:37:32.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:37:32.975 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:32.976 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.977 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.978 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.979 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.980 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:32.981 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.982 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:37:32.983 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.984 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.985 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:32.986 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:32.987 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:32.988 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:32.989 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:32.989 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:32.989 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:32.989 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:32.989 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.989 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:32.990 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.991 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:32.991 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:37:32.991 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:32.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:37:32.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:37:32.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:37:33.044 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:37:33.044 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:37:33.044 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.044 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.044 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:37:33.044 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:37:33.044 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.045 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.046 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.047 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 1, 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 1, 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.048 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0.019999812, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0.019999812 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 7, 2026-02-17T06:37:33.049 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 1, 2026-02-17T06:37:33.050 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:37:33.051 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.052 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.053 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.054 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 1, 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 1, 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 4.011960922, 2026-02-17T06:37:33.055 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 4.011960922 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 2, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.056 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 1, 2026-02-17T06:37:33.057 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.057 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:37:33.057 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0.003999958, 2026-02-17T06:37:33.057 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0.003999958 2026-02-17T06:37:33.057 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.058 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:37:33.059 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:33.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:37:33.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T06:37:33.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:37:33.112 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:37:33.112 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:37:33.112 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.112 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.112 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:37:33.112 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.113 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.114 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:33.115 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:33.116 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.117 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.118 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.119 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.120 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.121 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:33.122 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:37:33.123 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:37:33.124 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:37:33.125 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:37:33.125 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:37:33.125 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:37:33.125 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:37:33.125 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:37:33.126 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:37:33.127 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:37:33.127 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.127 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:37:33.127 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:37:33.127 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:33.127 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_dump_scrub_schedule ------------------ 2026-02-17T06:37:33.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_dump_scrub_schedule ------------------' 2026-02-17T06:37:33.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:37:33.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:37:33.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:37:33.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:37:33.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:37:33.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:37:33.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:37:33.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:37:33.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:37:33.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:37:33.429 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:37:33.430 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:37:33.431 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:37:33.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:37:33.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:37:33.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:37:33.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:37:33.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:37:33.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:37:33.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:37:33.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:37:33.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:37:33.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:37:33.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:37:33.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:37:33.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:37:33.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:37:33.453 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_dump_scrub_schedule ------------------ 2026-02-17T06:37:33.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_dump_scrub_schedule ------------------' 2026-02-17T06:37:33.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:37:33.454 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_interval_changes ------------------- 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_interval_changes -------------------' 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:37:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:37:33.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:37:33.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:37:33.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:37:33.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:37:33.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:37:33.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:37:33.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:37:33.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:37:33.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:37:33.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:37:33.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:37:33.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:37:33.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:37:33.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:37:33.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:37:33.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:37:33.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:37:33.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:37:33.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:37:33.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:37:33.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:37:33.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:37:33.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:37:33.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:37:33.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:37:33.471 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_interval_changes ----------------------- 2026-02-17T06:37:33.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_interval_changes -----------------------' 2026-02-17T06:37:33.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_interval_changes td/osd-scrub-test 2026-02-17T06:37:33.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:149: TEST_interval_changes: local poolname=test 2026-02-17T06:37:33.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:150: TEST_interval_changes: local OSDS=2 2026-02-17T06:37:33.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:151: TEST_interval_changes: local objects=10 2026-02-17T06:37:33.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:153: TEST_interval_changes: expr 24 '*' 60 '*' 60 2026-02-17T06:37:33.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:153: TEST_interval_changes: local day=86400 2026-02-17T06:37:33.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:154: TEST_interval_changes: expr 86400 '*' 7 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:154: TEST_interval_changes: local week=604800 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:155: TEST_interval_changes: local min_interval=86400 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:156: TEST_interval_changes: local max_interval=604800 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:157: TEST_interval_changes: local WAIT_FOR_UPDATE=15 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:159: TEST_interval_changes: TESTDATA=testdata.507902 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:162: TEST_interval_changes: run_mon td/osd-scrub-test a --osd_pool_default_size=2 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:37:33.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=2 2026-02-17T06:37:33.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:37:33.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:37:33.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:37:33.496 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:37:33.496 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.496 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:37:33.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T06:37:33.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:37:33.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:37:33.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:37:33.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:37:33.525 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:37:33.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:37:33.526 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:37:33.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:37:33.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:37:33.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:37:33.581 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:37:33.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:37:33.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:37:33.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:37:33.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:37:33.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:37:33.583 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:37:33.583 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.583 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:37:33.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:37:33.584 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:37:33.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:163: TEST_interval_changes: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:37:33.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:37:33.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:37:33.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:37:33.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:37:33.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:37:33.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:37:33.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:37:33.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:37:33.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:37:33.728 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:37:33.728 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.728 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:37:33.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:37:33.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:37:33.747 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: expr 2 - 1 2026-02-17T06:37:33.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: seq 0 1 2026-02-17T06:37:33.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:37:33.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:166: TEST_interval_changes: run_osd td/osd-scrub-test 0 --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T06:37:33.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:37:33.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:37:33.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:37:33.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0' 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:37:33.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:37:33.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=acd512de-1bdc-4e78-95e4-810f85d7a9b9 2026-02-17T06:37:33.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 acd512de-1bdc-4e78-95e4-810f85d7a9b9' 2026-02-17T06:37:33.757 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 acd512de-1bdc-4e78-95e4-810f85d7a9b9 2026-02-17T06:37:33.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:37:33.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCtDJRpyvlGLRAA8niNM267ZAHKA3qOkAyufA== 2026-02-17T06:37:33.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCtDJRpyvlGLRAA8niNM267ZAHKA3qOkAyufA=="}' 2026-02-17T06:37:33.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new acd512de-1bdc-4e78-95e4-810f85d7a9b9 -i td/osd-scrub-test/0/new.json 2026-02-17T06:37:33.855 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:37:33.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:37:33.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 --mkfs --key AQCtDJRpyvlGLRAA8niNM267ZAHKA3qOkAyufA== --osd-uuid acd512de-1bdc-4e78-95e4-810f85d7a9b9 2026-02-17T06:37:33.880 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:33.869+0000 7fb3be2958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:33.881 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:33.869+0000 7fb3be2958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:33.882 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:33.873+0000 7fb3be2958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:33.883 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:33.873+0000 7fb3be2958c0 -1 bdev(0x5615cb741c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:37:33.883 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:33.873+0000 7fb3be2958c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:37:36.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:37:36.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:37:36.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:37:36.940 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:37:36.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:37:37.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:37:37.121 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:37:37.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T06:37:37.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:37:37.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:37:37.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:37:37.134 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:37.125+0000 7f3925e428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:37.135 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:37.125+0000 7f3925e428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:37.136 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:37.125+0000 7f3925e428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:37.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:37:37.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:37:37.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:37:37.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:37:37.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:37:37.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:37.274 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:37:37.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:37:37.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:37.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:37:37.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:38.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:38.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:38.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:37:38.422 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:37:38.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:38.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:37:38.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:38.639 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:38.629+0000 7f3925e428c0 -1 Falling back to public interface 2026-02-17T06:37:39.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:39.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:39.570 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:37:39.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:37:39.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:39.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:37:39.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:40.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:40.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:40.720 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:37:40.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:37:40.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:40.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:37:40.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:41.658 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:41.649+0000 7f3925e428c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:37:41.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:41.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:41.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:37:41.876 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:37:41.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:41.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:37:42.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:43.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:43.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:43.028 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:37:43.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:37:43.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:43.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:37:43.172 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2556014662,v1:127.0.0.1:6803/2556014662] [v2:127.0.0.1:6804/2556014662,v1:127.0.0.1:6805/2556014662] exists,up acd512de-1bdc-4e78-95e4-810f85d7a9b9 2026-02-17T06:37:43.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:37:43.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:37:43.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:166: TEST_interval_changes: run_osd td/osd-scrub-test 1 --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:37:43.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:37:43.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:37:43.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0' 2026-02-17T06:37:43.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:37:43.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:37:43.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=64d2af7e-b9ff-4dc2-8547-fb723d73d013 2026-02-17T06:37:43.180 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 64d2af7e-b9ff-4dc2-8547-fb723d73d013 2026-02-17T06:37:43.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 64d2af7e-b9ff-4dc2-8547-fb723d73d013' 2026-02-17T06:37:43.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:37:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC3DJRpvdvfChAA4UlQ5i+rHxRi3QbskHz0Bg== 2026-02-17T06:37:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC3DJRpvdvfChAA4UlQ5i+rHxRi3QbskHz0Bg=="}' 2026-02-17T06:37:43.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 64d2af7e-b9ff-4dc2-8547-fb723d73d013 -i td/osd-scrub-test/1/new.json 2026-02-17T06:37:43.338 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:37:43.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:37:43.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 --mkfs --key AQC3DJRpvdvfChAA4UlQ5i+rHxRi3QbskHz0Bg== --osd-uuid 64d2af7e-b9ff-4dc2-8547-fb723d73d013 2026-02-17T06:37:43.362 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:43.353+0000 7fd80abd28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:43.364 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:43.353+0000 7fd80abd28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:43.365 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:43.353+0000 7fd80abd28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:43.365 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:43.353+0000 7fd80abd28c0 -1 bdev(0x56183fa5bc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:37:43.365 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:43.357+0000 7fd80abd28c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:37:45.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:37:45.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:37:45.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:37:45.653 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:37:45.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:37:45.844 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:37:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:37:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T06:37:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:37:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:37:45.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:37:45.866 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:45.857+0000 7f03ab8768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:45.868 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:45.857+0000 7f03ab8768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:45.869 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:45.857+0000 7f03ab8768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:37:45.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:37:45.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:37:45.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:37:45.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:37:45.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:37:45.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:45.996 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:37:45.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:37:45.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:45.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:37:46.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:47.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:47.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:47.147 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:37:47.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:37:47.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:47.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:37:47.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:47.347 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:47.337+0000 7f03ab8768c0 -1 Falling back to public interface 2026-02-17T06:37:48.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:48.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:48.293 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:37:48.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:37:48.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:48.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:37:48.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:49.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:49.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:49.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:37:49.443 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:37:49.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:49.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:37:49.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:50.381 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:37:50.369+0000 7f03ab8768c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:37:50.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:50.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:50.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:37:50.598 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:37:50.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:50.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:37:50.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:37:51.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:37:51.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:37:51.749 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:37:51.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:37:51.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:37:51.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:37:51.899 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1262981363,v1:127.0.0.1:6811/1262981363] [v2:127.0.0.1:6812/1262981363,v1:127.0.0.1:6813/1262981363] exists,up 64d2af7e-b9ff-4dc2-8547-fb723d73d013 2026-02-17T06:37:51.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:37:51.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:37:51.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:37:51.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: create_pool test 1 1 2026-02-17T06:37:51.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:37:52.094 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:37:52.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:37:53.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:171: TEST_interval_changes: wait_for_clean 2026-02-17T06:37:53.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:37:53.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:37:53.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:37:53.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:37:53.108 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:37:53.108 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:37:53.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:37:53.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:37:53.109 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:37:53.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:37:53.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:37:53.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:37:53.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:37:53.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:37:53.206 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:37:53.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:37:53.365 INFO:tasks.workunit.client.0.trial074.stderr:1' 2026-02-17T06:37:53.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:37:53.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:37:53.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:37:53.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T06:37:53.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T06:37:53.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T06:37:53.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:37:53.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:37:53.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705667 2026-02-17T06:37:53.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705667 2026-02-17T06:37:53.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705667' 2026-02-17T06:37:53.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:37:53.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T06:37:53.507 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:37:53.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:37:53.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T06:37:53.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:37:53.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T06:37:53.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T06:37:53.511 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836492 2026-02-17T06:37:53.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:37:53.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-17T06:37:53.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:37:54.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:37:54.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:37:54.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836492 2026-02-17T06:37:54.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:37:54.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705667 2026-02-17T06:37:54.812 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:37:54.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:37:54.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705667 2026-02-17T06:37:54.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:37:54.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705667 2026-02-17T06:37:54.816 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705667 2026-02-17T06:37:54.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-17T06:37:54.816 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:37:54.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-17T06:37:54.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:37:54.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:37:54.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:37:55.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:37:55.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:37:55.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:37:55.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:37:55.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:37:55.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:37:55.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:37:55.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:37:55.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:37:55.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:37:55.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:37:55.302 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:37:55.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:37:55.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:37:55.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:37:55.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: ceph osd dump 2026-02-17T06:37:55.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: grep '^pool.*['\'']test['\'']' 2026-02-17T06:37:55.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: awk '{ print $2 }' 2026-02-17T06:37:55.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: local poolid=1 2026-02-17T06:37:55.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:174: TEST_interval_changes: dd if=/dev/urandom of=testdata.507902 bs=1032 count=1 2026-02-17T06:37:55.630 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:37:55.630 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:37:55.630 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.4625e-05 s, 29.8 MB/s 2026-02-17T06:37:55.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: seq 1 10 2026-02-17T06:37:55.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj1 testdata.507902 2026-02-17T06:37:55.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj2 testdata.507902 2026-02-17T06:37:55.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj3 testdata.507902 2026-02-17T06:37:55.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj4 testdata.507902 2026-02-17T06:37:55.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj5 testdata.507902 2026-02-17T06:37:55.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj6 testdata.507902 2026-02-17T06:37:55.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj7 testdata.507902 2026-02-17T06:37:55.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj8 testdata.507902 2026-02-17T06:37:55.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj9 testdata.507902 2026-02-17T06:37:55.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T06:37:55.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj10 testdata.507902 2026-02-17T06:37:55.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:179: TEST_interval_changes: rm -f testdata.507902 2026-02-17T06:37:55.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:181: TEST_interval_changes: get_primary test obj1 2026-02-17T06:37:55.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:37:55.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:37:55.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:37:55.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:37:56.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:181: TEST_interval_changes: local primary=1 2026-02-17T06:37:56.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:184: TEST_interval_changes: check_dump_scrubs 1 '1 day' '1 week' 2026-02-17T06:37:56.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T06:37:56.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=1 day' 2026-02-17T06:37:56.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=1 week' 2026-02-17T06:37:56.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T06:37:56.004 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:37:56.005 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:37:56.005 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:37:56.005 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:56.005 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:56.005 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.1.asok 2026-02-17T06:37:56.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T06:37:56.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.507902/ceph-osd.1.asok dump_scrubs 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr: { 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr: "pgid": "1.0", 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr: "sched_time": "2026-02-18T06:37:52.088251+0000", 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr: "orig_sched_time": "2026-02-18T06:37:52.088251+0000", 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr: "deadline": "2026-02-24T06:37:52.088251+0000", 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr: "forced": false 2026-02-17T06:37:56.062 INFO:tasks.workunit.client.0.trial074.stderr: } 2026-02-17T06:37:56.063 INFO:tasks.workunit.client.0.trial074.stderr:]' 2026-02-17T06:37:56.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-18T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-18T06:37:52.088251+0000",' '"deadline":' '"2026-02-24T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:37:56.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T06:37:56.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-18T06:37:52.088251+0000"' 2026-02-17T06:37:56.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-18T06:37:52.088251+0000 2026-02-17T06:37:56.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-18T06:37:52.088251+0000 2026-02-17T06:37:56.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:37:56.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 1 day' 2026-02-17T06:37:56.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-18 = 2026-02-18 2026-02-17T06:37:56.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-18T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-18T06:37:52.088251+0000",' '"deadline":' '"2026-02-24T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:37:56.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T06:37:56.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-02-24T06:37:52.088251+0000"' 2026-02-17T06:37:56.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-02-24T06:37:52.088251+0000 2026-02-17T06:37:56.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-02-24T06:37:52.088251+0000 2026-02-17T06:37:56.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:37:56.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 1 week' 2026-02-17T06:37:56.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-02-24 = 2026-02-24 2026-02-17T06:37:56.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: get_asok_path osd.1 2026-02-17T06:37:56.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:37:56.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:37:56.092 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:37:56.092 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:37:56.092 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:37:56.092 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.1.asok 2026-02-17T06:37:56.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: expr 86400 '*' 2 2026-02-17T06:37:56.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: CEPH_ARGS= 2026-02-17T06:37:56.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: ceph --admin-daemon /tmp/ceph-asok.507902/ceph-osd.1.asok config set osd_scrub_min_interval 172800 2026-02-17T06:37:56.146 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:37:56.146 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-17T06:37:56.146 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:37:56.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:188: TEST_interval_changes: sleep 15 2026-02-17T06:38:11.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:189: TEST_interval_changes: check_dump_scrubs 1 '2 days' '1 week' 2026-02-17T06:38:11.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T06:38:11.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=2 days' 2026-02-17T06:38:11.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=1 week' 2026-02-17T06:38:11.155 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T06:38:11.155 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:38:11.155 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:38:11.155 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:38:11.155 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:11.155 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:11.156 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.1.asok 2026-02-17T06:38:11.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T06:38:11.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.507902/ceph-osd.1.asok dump_scrubs 2026-02-17T06:38:11.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr: { 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr: "pgid": "1.0", 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr: "sched_time": "2026-02-19T06:37:52.088251+0000", 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr: "orig_sched_time": "2026-02-19T06:37:52.088251+0000", 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr: "deadline": "2026-02-24T06:37:52.088251+0000", 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr: "forced": false 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr: } 2026-02-17T06:38:11.215 INFO:tasks.workunit.client.0.trial074.stderr:]' 2026-02-17T06:38:11.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"deadline":' '"2026-02-24T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:11.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T06:38:11.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-19T06:37:52.088251+0000"' 2026-02-17T06:38:11.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-19T06:37:52.088251+0000 2026-02-17T06:38:11.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-19T06:37:52.088251+0000 2026-02-17T06:38:11.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:11.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 days' 2026-02-17T06:38:11.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-19 = 2026-02-19 2026-02-17T06:38:11.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"deadline":' '"2026-02-24T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:11.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T06:38:11.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-02-24T06:37:52.088251+0000"' 2026-02-17T06:38:11.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-02-24T06:37:52.088251+0000 2026-02-17T06:38:11.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-02-24T06:37:52.088251+0000 2026-02-17T06:38:11.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:11.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 1 week' 2026-02-17T06:38:11.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-02-24 = 2026-02-24 2026-02-17T06:38:11.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: get_asok_path osd.1 2026-02-17T06:38:11.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:38:11.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:38:11.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:38:11.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:11.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:11.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.1.asok 2026-02-17T06:38:11.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: expr 604800 '*' 2 2026-02-17T06:38:11.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: CEPH_ARGS= 2026-02-17T06:38:11.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: ceph --admin-daemon /tmp/ceph-asok.507902/ceph-osd.1.asok config set osd_scrub_max_interval 1209600 2026-02-17T06:38:11.295 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:38:11.295 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_max_interval = '' " 2026-02-17T06:38:11.295 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:38:11.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:193: TEST_interval_changes: sleep 15 2026-02-17T06:38:26.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:194: TEST_interval_changes: check_dump_scrubs 1 '2 days' '2 week' 2026-02-17T06:38:26.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T06:38:26.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=2 days' 2026-02-17T06:38:26.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=2 week' 2026-02-17T06:38:26.304 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T06:38:26.304 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:38:26.304 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:38:26.304 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:38:26.304 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:26.305 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:26.305 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.1.asok 2026-02-17T06:38:26.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T06:38:26.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.507902/ceph-osd.1.asok dump_scrubs 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr: { 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr: "pgid": "1.0", 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr: "sched_time": "2026-02-19T06:37:52.088251+0000", 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr: "orig_sched_time": "2026-02-19T06:37:52.088251+0000", 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr: "deadline": "2026-03-03T06:37:52.088251+0000", 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr: "forced": false 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr: } 2026-02-17T06:38:26.362 INFO:tasks.workunit.client.0.trial074.stderr:]' 2026-02-17T06:38:26.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"deadline":' '"2026-03-03T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:26.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T06:38:26.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-19T06:37:52.088251+0000"' 2026-02-17T06:38:26.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-19T06:37:52.088251+0000 2026-02-17T06:38:26.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-19T06:37:52.088251+0000 2026-02-17T06:38:26.374 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:26.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 days' 2026-02-17T06:38:26.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-19 = 2026-02-19 2026-02-17T06:38:26.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-19T06:37:52.088251+0000",' '"deadline":' '"2026-03-03T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:26.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T06:38:26.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-03-03T06:37:52.088251+0000"' 2026-02-17T06:38:26.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-03-03T06:37:52.088251+0000 2026-02-17T06:38:26.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-03-03T06:37:52.088251+0000 2026-02-17T06:38:26.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:26.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 week' 2026-02-17T06:38:26.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-03-03 = 2026-03-03 2026-02-17T06:38:26.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:197: TEST_interval_changes: expr 86400 '*' 3 2026-02-17T06:38:26.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:197: TEST_interval_changes: ceph osd pool set test scrub_min_interval 259200 2026-02-17T06:38:26.587 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 scrub_min_interval to 259200 2026-02-17T06:38:26.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:198: TEST_interval_changes: sleep 15 2026-02-17T06:38:41.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:199: TEST_interval_changes: check_dump_scrubs 1 '3 days' '2 week' 2026-02-17T06:38:41.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T06:38:41.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=3 days' 2026-02-17T06:38:41.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=2 week' 2026-02-17T06:38:41.603 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T06:38:41.603 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:38:41.603 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:38:41.604 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:38:41.604 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:41.604 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:41.604 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.1.asok 2026-02-17T06:38:41.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T06:38:41.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.507902/ceph-osd.1.asok dump_scrubs 2026-02-17T06:38:41.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T06:38:41.663 INFO:tasks.workunit.client.0.trial074.stderr: { 2026-02-17T06:38:41.663 INFO:tasks.workunit.client.0.trial074.stderr: "pgid": "1.0", 2026-02-17T06:38:41.663 INFO:tasks.workunit.client.0.trial074.stderr: "sched_time": "2026-02-20T06:37:52.088251+0000", 2026-02-17T06:38:41.664 INFO:tasks.workunit.client.0.trial074.stderr: "orig_sched_time": "2026-02-20T06:37:52.088251+0000", 2026-02-17T06:38:41.664 INFO:tasks.workunit.client.0.trial074.stderr: "deadline": "2026-03-03T06:37:52.088251+0000", 2026-02-17T06:38:41.664 INFO:tasks.workunit.client.0.trial074.stderr: "forced": false 2026-02-17T06:38:41.664 INFO:tasks.workunit.client.0.trial074.stderr: } 2026-02-17T06:38:41.664 INFO:tasks.workunit.client.0.trial074.stderr:]' 2026-02-17T06:38:41.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"deadline":' '"2026-03-03T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:41.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T06:38:41.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-20T06:37:52.088251+0000"' 2026-02-17T06:38:41.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-20T06:37:52.088251+0000 2026-02-17T06:38:41.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-20T06:37:52.088251+0000 2026-02-17T06:38:41.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:41.676 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 days' 2026-02-17T06:38:41.677 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-20 = 2026-02-20 2026-02-17T06:38:41.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"deadline":' '"2026-03-03T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:41.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T06:38:41.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-03-03T06:37:52.088251+0000"' 2026-02-17T06:38:41.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-03-03T06:37:52.088251+0000 2026-02-17T06:38:41.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-03-03T06:37:52.088251+0000 2026-02-17T06:38:41.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:41.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 week' 2026-02-17T06:38:41.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-03-03 = 2026-03-03 2026-02-17T06:38:41.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:202: TEST_interval_changes: expr 604800 '*' 3 2026-02-17T06:38:41.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:202: TEST_interval_changes: ceph osd pool set test scrub_max_interval 1814400 2026-02-17T06:38:41.884 INFO:tasks.workunit.client.0.trial074.stderr:set pool 1 scrub_max_interval to 1814400 2026-02-17T06:38:41.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:203: TEST_interval_changes: sleep 15 2026-02-17T06:38:56.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:204: TEST_interval_changes: check_dump_scrubs 1 '3 days' '3 week' 2026-02-17T06:38:56.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T06:38:56.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=3 days' 2026-02-17T06:38:56.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=3 week' 2026-02-17T06:38:56.898 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T06:38:56.899 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T06:38:56.899 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T06:38:56.899 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:38:56.899 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:56.899 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:56.899 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.1.asok 2026-02-17T06:38:56.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T06:38:56.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.507902/ceph-osd.1.asok dump_scrubs 2026-02-17T06:38:56.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T06:38:56.958 INFO:tasks.workunit.client.0.trial074.stderr: { 2026-02-17T06:38:56.958 INFO:tasks.workunit.client.0.trial074.stderr: "pgid": "1.0", 2026-02-17T06:38:56.958 INFO:tasks.workunit.client.0.trial074.stderr: "sched_time": "2026-02-20T06:37:52.088251+0000", 2026-02-17T06:38:56.959 INFO:tasks.workunit.client.0.trial074.stderr: "orig_sched_time": "2026-02-20T06:37:52.088251+0000", 2026-02-17T06:38:56.959 INFO:tasks.workunit.client.0.trial074.stderr: "deadline": "2026-03-10T06:37:52.088251+0000", 2026-02-17T06:38:56.959 INFO:tasks.workunit.client.0.trial074.stderr: "forced": false 2026-02-17T06:38:56.959 INFO:tasks.workunit.client.0.trial074.stderr: } 2026-02-17T06:38:56.959 INFO:tasks.workunit.client.0.trial074.stderr:]' 2026-02-17T06:38:56.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"deadline":' '"2026-03-10T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:56.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T06:38:56.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-20T06:37:52.088251+0000"' 2026-02-17T06:38:56.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-20T06:37:52.088251+0000 2026-02-17T06:38:56.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-20T06:37:52.088251+0000 2026-02-17T06:38:56.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:56.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 days' 2026-02-17T06:38:56.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-20 = 2026-02-20 2026-02-17T06:38:56.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"orig_sched_time":' '"2026-02-20T06:37:52.088251+0000",' '"deadline":' '"2026-03-10T06:37:52.088251+0000",' '"forced":' false '}' ']' 2026-02-17T06:38:56.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T06:38:56.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-03-10T06:37:52.088251+0000"' 2026-02-17T06:38:56.985 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-03-10T06:37:52.088251+0000 2026-02-17T06:38:56.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-03-10T06:37:52.088251+0000 2026-02-17T06:38:56.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T06:38:56.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 week' 2026-02-17T06:38:56.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-03-10 = 2026-03-10 2026-02-17T06:38:56.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:205: TEST_interval_changes: perf_counters td/osd-scrub-test 2 2026-02-17T06:38:56.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:38:56.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=2 2026-02-17T06:38:56.990 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 2 - 1 2026-02-17T06:38:56.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 1 2026-02-17T06:38:56.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:38:56.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:38:56.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.064 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:38:57.065 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.066 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.067 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.068 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:38:57.070 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:38:57.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:38:57.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:38:57.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.132 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.133 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.134 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.135 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.136 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.137 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.138 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.139 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.140 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.141 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.142 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.143 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:38:57.144 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:38:57.145 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.145 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:38:57.146 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:38:57.147 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.147 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:38:57.147 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:38:57.147 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:38:57.147 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_interval_changes ------------------ 2026-02-17T06:38:57.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_interval_changes ------------------' 2026-02-17T06:38:57.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:38:57.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:38:57.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:38:57.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:38:57.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:38:57.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:38:57.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:38:57.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:38:57.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:38:57.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:38:57.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:38:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:38:57.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:38:57.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:38:57.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:38:57.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:38:57.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:38:57.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:38:57.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:38:57.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:38:57.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:38:57.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:38:57.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:38:57.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:38:57.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:57.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:57.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:38:57.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:38:57.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:38:57.467 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_interval_changes ------------------ 2026-02-17T06:38:57.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_interval_changes ------------------' 2026-02-17T06:38:57.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:38:57.467 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_just_deep_scrubs ------------------- 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_just_deep_scrubs -------------------' 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:38:57.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:38:57.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:38:57.470 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:38:57.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:38:57.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:38:57.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:38:57.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:38:57.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:38:57.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:38:57.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:38:57.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:38:57.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:38:57.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:38:57.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:38:57.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:38:57.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:57.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:57.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:38:57.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:38:57.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:38:57.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:38:57.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:38:57.481 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:38:57.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:38:57.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:38:57.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:38:57.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:38:57.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:38:57.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:38:57.484 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_just_deep_scrubs ----------------------- 2026-02-17T06:38:57.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_just_deep_scrubs -----------------------' 2026-02-17T06:38:57.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_just_deep_scrubs td/osd-scrub-test 2026-02-17T06:38:57.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:467: TEST_just_deep_scrubs: local dir=td/osd-scrub-test 2026-02-17T06:38:57.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:468: TEST_just_deep_scrubs: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='4' ['pool_name']='test') 2026-02-17T06:38:57.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:468: TEST_just_deep_scrubs: local -A cluster_conf 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:474: TEST_just_deep_scrubs: standard_scrub_cluster td/osd-scrub-test cluster_conf 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-test 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:234: standard_scrub_cluster: local poolname=test 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: args['pool_name']=test 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: local extra_pars= 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:237: standard_scrub_cluster: local debug_msg=dbg 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:240: standard_scrub_cluster: local saved_echo_flag=x 2026-02-17T06:38:57.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: set +x 2026-02-17T06:38:57.760 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 1411a729-503a-452f-aa6b-8a897f50fc83 2026-02-17T06:38:57.858 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:38:57.883 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:38:57.876+0000 7fbc036d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:38:57.885 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:38:57.876+0000 7fbc036d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:38:57.886 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:38:57.880+0000 7fbc036d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:38:57.886 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:38:57.880+0000 7fbc036d88c0 -1 bdev(0x55588ba4fc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:38:57.886 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:38:57.880+0000 7fbc036d88c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:39:00.760 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:39:00.950 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:39:00.974 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:00.968+0000 7fae3289d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:00.975 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:00.968+0000 7fae3289d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:00.976 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:00.968+0000 7fae3289d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:01.101 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:39:02.256 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:39:02.475 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:02.468+0000 7fae3289d8c0 -1 Falling back to public interface 2026-02-17T06:39:03.410 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:39:04.561 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:39:05.474 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:05.468+0000 7fae3289d8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:39:05.713 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:39:06.863 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:39:07.012 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4205840986,v1:127.0.0.1:6803/4205840986] [v2:127.0.0.1:6804/4205840986,v1:127.0.0.1:6805/4205840986] exists,up 1411a729-503a-452f-aa6b-8a897f50fc83 2026-02-17T06:39:07.017 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 d6166489-0252-43c8-af8a-130af9e1187d 2026-02-17T06:39:07.189 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:39:07.216 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:07.208+0000 7fcc4aa858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:07.217 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:07.212+0000 7fcc4aa858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:07.218 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:07.212+0000 7fcc4aa858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:07.219 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:07.212+0000 7fcc4aa858c0 -1 bdev(0x5647579b7c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:39:07.219 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:07.212+0000 7fcc4aa858c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:39:09.499 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:39:09.693 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:39:09.711 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:09.704+0000 7ff4078bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:09.712 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:09.704+0000 7ff4078bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:09.713 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:09.704+0000 7ff4078bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:09.846 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:39:10.997 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:39:11.219 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:11.212+0000 7ff4078bc8c0 -1 Falling back to public interface 2026-02-17T06:39:12.150 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:39:13.302 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:39:14.192 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:14.184+0000 7ff4078bc8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:39:14.452 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:39:15.604 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:39:15.754 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1979426961,v1:127.0.0.1:6811/1979426961] [v2:127.0.0.1:6812/1979426961,v1:127.0.0.1:6813/1979426961] exists,up d6166489-0252-43c8-af8a-130af9e1187d 2026-02-17T06:39:15.759 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 c00ef695-5236-4c42-ae10-87bf81a55ebb 2026-02-17T06:39:15.929 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:39:15.955 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:15.948+0000 7fbef97af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:15.956 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:15.948+0000 7fbef97af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:15.957 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:15.948+0000 7fbef97af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:15.957 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:15.952+0000 7fbef97af8c0 -1 bdev(0x55c3e652fc00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:39:15.958 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:15.952+0000 7fbef97af8c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:39:18.249 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:39:18.435 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:39:18.459 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:18.452+0000 7f893087e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:18.461 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:18.452+0000 7f893087e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:18.462 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:18.456+0000 7f893087e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:18.588 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:39:19.738 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:39:19.943 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:19.936+0000 7f893087e8c0 -1 Falling back to public interface 2026-02-17T06:39:20.890 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:39:22.038 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:39:22.933 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:22.924+0000 7f893087e8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:39:23.186 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:39:24.337 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:39:24.485 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2120664617,v1:127.0.0.1:6819/2120664617] [v2:127.0.0.1:6820/2120664617,v1:127.0.0.1:6821/2120664617] exists,up c00ef695-5236-4c42-ae10-87bf81a55ebb 2026-02-17T06:39:24.680 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:39:26.162 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T06:39:28.620 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T06:39:28.783 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509442 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stdout:standard_scrub_cluster: dbg: test pool is test 1 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:475: TEST_just_deep_scrubs: local poolid=1 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:476: TEST_just_deep_scrubs: local poolname=test 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stdout:Pool: test : 1 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: echo 'Pool: test : 1' 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: TESTDATA=testdata.507902 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:480: TEST_just_deep_scrubs: local objects=90 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:481: TEST_just_deep_scrubs: dd if=/dev/urandom of=testdata.507902 bs=1032 count=1 2026-02-17T06:39:30.585 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:39:30.586 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:39:30.586 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.8462e-05 s, 26.8 MB/s 2026-02-17T06:39:30.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: seq 1 90 2026-02-17T06:39:30.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.587 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj1 testdata.507902 2026-02-17T06:39:30.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj2 testdata.507902 2026-02-17T06:39:30.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj3 testdata.507902 2026-02-17T06:39:30.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj4 testdata.507902 2026-02-17T06:39:30.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj5 testdata.507902 2026-02-17T06:39:30.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj6 testdata.507902 2026-02-17T06:39:30.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj7 testdata.507902 2026-02-17T06:39:30.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj8 testdata.507902 2026-02-17T06:39:30.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj9 testdata.507902 2026-02-17T06:39:30.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj10 testdata.507902 2026-02-17T06:39:30.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.817 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj11 testdata.507902 2026-02-17T06:39:30.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj12 testdata.507902 2026-02-17T06:39:30.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj13 testdata.507902 2026-02-17T06:39:30.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj14 testdata.507902 2026-02-17T06:39:30.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj15 testdata.507902 2026-02-17T06:39:30.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj16 testdata.507902 2026-02-17T06:39:30.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj17 testdata.507902 2026-02-17T06:39:30.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:30.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj18 testdata.507902 2026-02-17T06:39:31.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj19 testdata.507902 2026-02-17T06:39:31.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj20 testdata.507902 2026-02-17T06:39:31.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj21 testdata.507902 2026-02-17T06:39:31.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj22 testdata.507902 2026-02-17T06:39:31.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj23 testdata.507902 2026-02-17T06:39:31.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj24 testdata.507902 2026-02-17T06:39:31.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj25 testdata.507902 2026-02-17T06:39:31.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj26 testdata.507902 2026-02-17T06:39:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj27 testdata.507902 2026-02-17T06:39:31.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj28 testdata.507902 2026-02-17T06:39:31.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj29 testdata.507902 2026-02-17T06:39:31.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj30 testdata.507902 2026-02-17T06:39:31.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj31 testdata.507902 2026-02-17T06:39:31.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj32 testdata.507902 2026-02-17T06:39:31.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj33 testdata.507902 2026-02-17T06:39:31.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj34 testdata.507902 2026-02-17T06:39:31.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj35 testdata.507902 2026-02-17T06:39:31.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj36 testdata.507902 2026-02-17T06:39:31.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj37 testdata.507902 2026-02-17T06:39:31.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj38 testdata.507902 2026-02-17T06:39:31.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj39 testdata.507902 2026-02-17T06:39:31.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj40 testdata.507902 2026-02-17T06:39:31.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj41 testdata.507902 2026-02-17T06:39:31.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj42 testdata.507902 2026-02-17T06:39:31.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj43 testdata.507902 2026-02-17T06:39:31.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj44 testdata.507902 2026-02-17T06:39:31.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj45 testdata.507902 2026-02-17T06:39:31.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj46 testdata.507902 2026-02-17T06:39:31.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj47 testdata.507902 2026-02-17T06:39:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj48 testdata.507902 2026-02-17T06:39:31.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj49 testdata.507902 2026-02-17T06:39:31.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj50 testdata.507902 2026-02-17T06:39:31.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj51 testdata.507902 2026-02-17T06:39:31.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj52 testdata.507902 2026-02-17T06:39:31.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj53 testdata.507902 2026-02-17T06:39:31.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj54 testdata.507902 2026-02-17T06:39:31.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj55 testdata.507902 2026-02-17T06:39:31.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj56 testdata.507902 2026-02-17T06:39:31.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj57 testdata.507902 2026-02-17T06:39:31.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj58 testdata.507902 2026-02-17T06:39:31.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj59 testdata.507902 2026-02-17T06:39:31.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj60 testdata.507902 2026-02-17T06:39:31.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:31.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj61 testdata.507902 2026-02-17T06:39:32.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj62 testdata.507902 2026-02-17T06:39:32.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj63 testdata.507902 2026-02-17T06:39:32.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj64 testdata.507902 2026-02-17T06:39:32.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj65 testdata.507902 2026-02-17T06:39:32.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj66 testdata.507902 2026-02-17T06:39:32.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj67 testdata.507902 2026-02-17T06:39:32.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj68 testdata.507902 2026-02-17T06:39:32.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj69 testdata.507902 2026-02-17T06:39:32.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj70 testdata.507902 2026-02-17T06:39:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj71 testdata.507902 2026-02-17T06:39:32.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj72 testdata.507902 2026-02-17T06:39:32.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj73 testdata.507902 2026-02-17T06:39:32.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj74 testdata.507902 2026-02-17T06:39:32.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj75 testdata.507902 2026-02-17T06:39:32.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj76 testdata.507902 2026-02-17T06:39:32.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj77 testdata.507902 2026-02-17T06:39:32.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj78 testdata.507902 2026-02-17T06:39:32.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj79 testdata.507902 2026-02-17T06:39:32.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj80 testdata.507902 2026-02-17T06:39:32.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj81 testdata.507902 2026-02-17T06:39:32.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj82 testdata.507902 2026-02-17T06:39:32.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj83 testdata.507902 2026-02-17T06:39:32.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj84 testdata.507902 2026-02-17T06:39:32.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj85 testdata.507902 2026-02-17T06:39:32.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj86 testdata.507902 2026-02-17T06:39:32.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj87 testdata.507902 2026-02-17T06:39:32.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj88 testdata.507902 2026-02-17T06:39:32.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj89 testdata.507902 2026-02-17T06:39:32.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T06:39:32.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj90 testdata.507902 2026-02-17T06:39:32.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:486: TEST_just_deep_scrubs: rm -f testdata.507902 2026-02-17T06:39:32.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:491: TEST_just_deep_scrubs: ceph osd set noscrub 2026-02-17T06:39:32.911 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:39:32.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:492: TEST_just_deep_scrubs: ceph osd set nodeep-scrub 2026-02-17T06:39:33.111 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is set 2026-02-17T06:39:33.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:493: TEST_just_deep_scrubs: sleep 6 2026-02-17T06:39:39.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:494: TEST_just_deep_scrubs: date -Ins 2026-02-17T06:39:39.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:494: TEST_just_deep_scrubs: local now_is=2026-02-17T06:39:39,118939734+00:00 2026-02-17T06:39:39.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:495: TEST_just_deep_scrubs: declare -A sched_data 2026-02-17T06:39:39.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:496: TEST_just_deep_scrubs: local pgid=1.2 2026-02-17T06:39:39.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:499: TEST_just_deep_scrubs: set_query_debug 1.2 2026-02-17T06:39:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:303: set_query_debug: local pgid=1.2 2026-02-17T06:39:39.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: ceph pg dump pgs_brief 2026-02-17T06:39:39.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: awk -v 'pg=^1.2' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-17T06:39:39.263 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs_brief 2026-02-17T06:39:39.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: local prim_osd=0 2026-02-17T06:39:39.273 INFO:tasks.workunit.client.0.trial074.stdout:Setting scrub debug data. Primary for 1.2 is 0 2026-02-17T06:39:39.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:307: set_query_debug: echo 'Setting scrub debug data. Primary for 1.2 is 0' 2026-02-17T06:39:39.274 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: get_asok_path osd.0 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-osd.0.asok 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: CEPH_ARGS= 2026-02-17T06:39:39.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.507902/ceph-osd.0.asok scrubdebug 1.2 set sessions 2026-02-17T06:39:39.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:501: TEST_just_deep_scrubs: extract_published_sch 1.2 2026-02-17T06:39:39,118939734+00:00 2026-02-17T06:39:39,118939734+00:00 sched_data 2026-02-17T06:39:39.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.2 2026-02-17T06:39:39.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sched_data 2026-02-17T06:39:39.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-17T06:39:39,118939734+00:00 2026-02-17T06:39:39.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2026-02-17T06:39:39,118939734+00:00 2026-02-17T06:39:39.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-17T06:39:39.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-17T06:39:39.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-17T06:39:39.477 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout:{"success":true}from pg dump pg: { 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 0, 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-18T06:39:24.674761+0000", 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 21, 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 38 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:39:39.486 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:39:39.559 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-18T06:39:24.674761+0000", 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-18T06:39:24.674 (2026-02-18T06:39:24.674)", 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "test_sequence": 5550592 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:39.560 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:39:39.640 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=21 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=39 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-18T06:39:24.674' 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-18T06:39:24.674' 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=0 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-17T06:39:24.674761+0000' 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='0x0' 2026-02-17T06:39:39.641 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=true 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=5550592 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=0 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-18T06:39:24.674761+0000' 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=21 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=38 2026-02-17T06:39:39.642 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:39:39.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:502: TEST_just_deep_scrubs: local saved_last_stamp=2026-02-17T06:39:24.674761+0000 2026-02-17T06:39:39.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:503: TEST_just_deep_scrubs: local dbg_counter_at_start=5550592 2026-02-17T06:39:39.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:504: TEST_just_deep_scrubs: echo 'test counter @ start: 5550592' 2026-02-17T06:39:39.652 INFO:tasks.workunit.client.0.trial074.stdout:test counter @ start: 5550592 2026-02-17T06:39:39.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:506: TEST_just_deep_scrubs: ceph tell 1.2 schedule-deep-scrub 2026-02-17T06:39:39.712 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:39:39.712 INFO:tasks.workunit.client.0.trial074.stdout: "deep": true, 2026-02-17T06:39:39.712 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:39:39.712 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-03T06:37:59.707003+0000" 2026-02-17T06:39:39.712 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:39:39.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:508: TEST_just_deep_scrubs: sleep 5 2026-02-17T06:39:44.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:510: TEST_just_deep_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-17T06:39:44.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:510: TEST_just_deep_scrubs: jq -r '.pg_stats[] | "\(.pgid) \(.stat_sum.num_objects)"' 2026-02-17T06:39:44.857 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:39:44.866 INFO:tasks.workunit.client.0.trial074.stdout:1.3 20 2026-02-17T06:39:44.866 INFO:tasks.workunit.client.0.trial074.stdout:1.2 20 2026-02-17T06:39:44.866 INFO:tasks.workunit.client.0.trial074.stdout:1.1 25 2026-02-17T06:39:44.866 INFO:tasks.workunit.client.0.trial074.stdout:1.0 25 2026-02-17T06:39:44.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:511: TEST_just_deep_scrubs: ceph pg 1.2 query --format=json-pretty 2026-02-17T06:39:44.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:511: TEST_just_deep_scrubs: jq -r .info.stats.stat_sum.num_objects 2026-02-17T06:39:44.942 INFO:tasks.workunit.client.0.trial074.stdout:Objects # in pg 1.2: 20 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:511: TEST_just_deep_scrubs: echo 'Objects # in pg 1.2: ' 20 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:513: TEST_just_deep_scrubs: declare -A sc_data_2 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:514: TEST_just_deep_scrubs: extract_published_sch 1.2 2026-02-17T06:39:39,118939734+00:00 2026-02-17T06:39:39,118939734+00:00 sc_data_2 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.2 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sc_data_2 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-17T06:39:39,118939734+00:00 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2026-02-17T06:39:39,118939734+00:00 2026-02-17T06:39:44.944 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-17T06:39:44.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-17T06:39:44.945 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-17T06:39:45.079 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 0, 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "queued for deep scrub", 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "0", 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": false, 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": false, 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 21, 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 40 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:39:45.089 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-17T06:39:45.225530+0000", 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "deep scrub scheduled @ 2026-02-17T06:39:45.225 (2026-02-04T06:37:59.707)", 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "test_sequence": 5550592 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:45.159 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=21 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=41 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='deep scrub scheduled' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-17T06:39:45.225' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-04T06:37:59.707' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=0 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-03T06:37:59.707003+0000' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='0x0' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=false 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=5550592 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=0 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='queued for deep scrub' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='0' 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=false 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=false 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=21 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=40 2026-02-17T06:39:45.242 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:39:45.253 INFO:tasks.workunit.client.0.trial074.stdout:test counter @ should show no change: 5550592 2026-02-17T06:39:45.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:515: TEST_just_deep_scrubs: echo 'test counter @ should show no change: ' 5550592 2026-02-17T06:39:45.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:516: TEST_just_deep_scrubs: (( 0 == 0 )) 2026-02-17T06:39:45.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:517: TEST_just_deep_scrubs: (( 5550592 == 5550592 )) 2026-02-17T06:39:45.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:520: TEST_just_deep_scrubs: ceph osd unset nodeep-scrub 2026-02-17T06:39:45.447 INFO:tasks.workunit.client.0.trial074.stderr:nodeep-scrub is unset 2026-02-17T06:39:45.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:521: TEST_just_deep_scrubs: sleep 5 2026-02-17T06:39:50.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:522: TEST_just_deep_scrubs: expct_qry_duration=(['query_last_duration']='0' ['query_last_duration_neg']='not0') 2026-02-17T06:39:50.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:522: TEST_just_deep_scrubs: declare -A expct_qry_duration 2026-02-17T06:39:50.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:523: TEST_just_deep_scrubs: sc_data_2=() 2026-02-17T06:39:50.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:524: TEST_just_deep_scrubs: echo 'test counter @ should be higher than before the unset: ' 2026-02-17T06:39:50.459 INFO:tasks.workunit.client.0.trial074.stdout:test counter @ should be higher than before the unset: 2026-02-17T06:39:50.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:525: TEST_just_deep_scrubs: wait_any_cond 1.2 10 2026-02-17T06:39:24.674761+0000 expct_qry_duration 'WaitingAfterScrub ' sc_data_2 2026-02-17T06:39:50.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.2 2026-02-17T06:39:50.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T06:39:50.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T06:39:24.674761+0000 2026-02-17T06:39:50.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_qry_duration 2026-02-17T06:39:50.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sc_data_2 2026-02-17T06:39:50.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T06:39:50.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T06:39:50.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T06:39:50.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T06:39:50.461 INFO:tasks.workunit.client.0.trial074.stdout:waiting for any condition (WaitingAfterScrub ): pg:1.2 dt:2026-02-17T06:39:24.674761+0000 (10 retries) 2026-02-17T06:39:51.103 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:39:51.113 INFO:tasks.workunit.client.0.trial074.stdout:from pg dump pg: { 2026-02-17T06:39:51.113 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_pg_state": "active+clean", 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_last_duration": 0, 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule": "periodic deep scrub scheduled", 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_schedule_at": "2026-02-17T06:39:50.262333+0000", 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_is_future": true, 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_vs_date": true, 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_reported_epoch": 22, 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout: "dmp_seq": 42 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:39:51.114 INFO:tasks.workunit.client.0.trial074.stdout:query output: 2026-02-17T06:39:51.184 INFO:tasks.workunit.client.0.trial074.stdout: "scrubber": { 2026-02-17T06:39:51.184 INFO:tasks.workunit.client.0.trial074.stdout: "active": false, 2026-02-17T06:39:51.184 INFO:tasks.workunit.client.0.trial074.stdout: "must_scrub": false, 2026-02-17T06:39:51.184 INFO:tasks.workunit.client.0.trial074.stdout: "must_deep_scrub": false, 2026-02-17T06:39:51.184 INFO:tasks.workunit.client.0.trial074.stdout: "must_repair": false, 2026-02-17T06:39:51.184 INFO:tasks.workunit.client.0.trial074.stdout: "need_auto": false, 2026-02-17T06:39:51.184 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reg_stamp": "2026-02-18T06:39:50.293503+0000", 2026-02-17T06:39:51.185 INFO:tasks.workunit.client.0.trial074.stdout: "schedule": "scrub scheduled @ 2026-02-18T06:39:50.293 (2026-02-18T06:39:50.293)", 2026-02-17T06:39:51.185 INFO:tasks.workunit.client.0.trial074.stdout: "test_sequence": 5550595 2026-02-17T06:39:51.185 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.185 INFO:tasks.workunit.client.0.trial074.stdout: "agent_state": {} 2026-02-17T06:39:51.265 INFO:tasks.workunit.client.0.trial074.stdout:( 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_epoch]=22 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_seq]=52 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_active]=false 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule]='scrub scheduled' 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_schedule_at]='2026-02-18T06:39:50.293' 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_target_at]='2026-02-18T06:39:50.293' 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_duration]=1 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_stamp]='2026-02-17T06:39:50.293503+0000' 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_last_scrub]='19x20' 2026-02-17T06:39:51.266 INFO:tasks.workunit.client.0.trial074.stdout:[query_is_future]=true 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[query_vs_date]=true 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[query_scrub_seq]=5550595 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_pg_state]='active+clean' 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_last_duration]=0 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule]='periodic deep scrub scheduled' 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_schedule_at]='2026-02-17T06:39:50.262333+0000' 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_is_future]=true 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_vs_date]=true 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_reported_epoch]=22 2026-02-17T06:39:51.267 INFO:tasks.workunit.client.0.trial074.stdout:[dmp_seq]=42 2026-02-17T06:39:51.268 INFO:tasks.workunit.client.0.trial074.stdout:) 2026-02-17T06:39:51.278 INFO:tasks.workunit.client.0.trial074.stdout:--> loop: 1 ~ false / 52 / 42 / true / 2026-02-17T06:39:50.293503+0000 / scrub scheduled %%% query_last_duration_neg query_last_duration 2026-02-17T06:39:51.278 INFO:tasks.workunit.client.0.trial074.stdout:key is query_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-17T06:39:51.278 INFO:tasks.workunit.client.0.trial074.stdout:WaitingAfterScrub - 'query_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-17T06:39:51.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T06:39:51.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:526: TEST_just_deep_scrubs: perf_counters td/osd-scrub-test 3 2026-02-17T06:39:51.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:39:51.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T06:39:51.279 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T06:39:51.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T06:39:51.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:39:51.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:39:51.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.352 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 1, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 1, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0.007999911, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0.007999911 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 2, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 1, 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.353 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.354 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.355 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.356 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.356 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.356 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.360 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:39:51.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:39:51.423 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.424 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.425 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.426 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:39:51.427 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.428 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.429 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.430 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:39:51.431 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.432 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.433 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.434 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.435 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.436 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.437 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.437 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.437 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.437 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.437 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.437 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:39:51.438 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:39:51.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:39:51.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T06:39:51.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:39:51.494 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:39:51.494 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.495 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.496 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.497 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.498 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.499 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.500 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.501 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:39:51.502 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.503 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.504 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.505 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:39:51.506 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:39:51.507 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:39:51.508 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:39:51.508 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:39:51.508 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:39:51.508 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:39:51.508 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:39:51.509 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:39:51.510 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_just_deep_scrubs ------------------ 2026-02-17T06:39:51.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_just_deep_scrubs ------------------' 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:39:51.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:39:51.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:39:51.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:39:51.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:39:51.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:39:51.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:39:51.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:39:51.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:39:51.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:39:51.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:39:51.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:39:51.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:39:51.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:39:51.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:39:51.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:39:51.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:39:51.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:39:51.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:39:51.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:39:51.635 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_just_deep_scrubs ------------------ 2026-02-17T06:39:51.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_just_deep_scrubs ------------------' 2026-02-17T06:39:51.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:39:51.636 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_pg_dump_objects_scrubbed ------------------- 2026-02-17T06:39:51.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_pg_dump_objects_scrubbed -------------------' 2026-02-17T06:39:51.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:39:51.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:39:51.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:39:51.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:39:51.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:39:51.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:39:51.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:39:51.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:39:51.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:39:51.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:39:51.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:39:51.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:39:51.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:39:51.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:39:51.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:39:51.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:39:51.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:39:51.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:39:51.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:39:51.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:39:51.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:39:51.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:39:51.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:39:51.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:39:51.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.650 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:39:51.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:39:51.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:39:51.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:39:51.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:39:51.653 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_pg_dump_objects_scrubbed ----------------------- 2026-02-17T06:39:51.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_pg_dump_objects_scrubbed -----------------------' 2026-02-17T06:39:51.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_pg_dump_objects_scrubbed td/osd-scrub-test 2026-02-17T06:39:51.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:651: TEST_pg_dump_objects_scrubbed: local dir=td/osd-scrub-test 2026-02-17T06:39:51.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:652: TEST_pg_dump_objects_scrubbed: local poolname=test 2026-02-17T06:39:51.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:653: TEST_pg_dump_objects_scrubbed: local OSDS=3 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:654: TEST_pg_dump_objects_scrubbed: local objects=15 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:655: TEST_pg_dump_objects_scrubbed: local timeout=10 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:657: TEST_pg_dump_objects_scrubbed: TESTDATA=testdata.507902 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:659: TEST_pg_dump_objects_scrubbed: setup td/osd-scrub-test 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:39:51.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:39:51.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:39:51.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:39:51.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:39:51.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:39:51.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:39:51.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:39:51.658 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:39:51.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:39:51.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:39:51.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:39:51.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:39:51.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:39:51.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:39:51.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:39:51.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:39:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:39:51.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:39:51.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:39:51.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.665 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:39:51.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:39:51.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:39:51.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:39:51.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:39:51.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.668 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:39:51.669 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:39:51.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:660: TEST_pg_dump_objects_scrubbed: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:39:51.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T06:39:51.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:39:51.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:39:51.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:39:51.692 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:39:51.692 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.692 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:39:51.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T06:39:51.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:39:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:39:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:39:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:39:51.721 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:39:51.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:39:51.722 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:39:51.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:39:51.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:39:51.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:39:51.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:39:51.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:39:51.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:39:51.780 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:39:51.780 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:39:51.780 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:39:51.780 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:39:51.780 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.780 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.781 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:39:51.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:39:51.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:39:51.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:661: TEST_pg_dump_objects_scrubbed: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:39:51.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:39:51.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:39:51.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:39:51.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:39:51.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:39:51.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:39:51.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:39:51.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:39:51.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:39:51.927 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:39:51.927 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.927 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:39:51.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:39:51.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:39:51.946 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: expr 3 - 1 2026-02-17T06:39:51.947 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: seq 0 2 2026-02-17T06:39:51.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:39:51.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 0 2026-02-17T06:39:51.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:39:51.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:39:51.953 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:39:51.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:39:51.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:39:51.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:39:51.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:39:51.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:39:51.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c328d2ca-818b-4cc0-b5a2-025d05e4b2e2 2026-02-17T06:39:51.956 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 c328d2ca-818b-4cc0-b5a2-025d05e4b2e2 2026-02-17T06:39:51.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 c328d2ca-818b-4cc0-b5a2-025d05e4b2e2' 2026-02-17T06:39:51.956 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:39:51.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA3DZRpuQcFORAA3q2IFhQgPCOLrCbSj9U2yw== 2026-02-17T06:39:51.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA3DZRpuQcFORAA3q2IFhQgPCOLrCbSj9U2yw=="}' 2026-02-17T06:39:51.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c328d2ca-818b-4cc0-b5a2-025d05e4b2e2 -i td/osd-scrub-test/0/new.json 2026-02-17T06:39:52.050 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:39:52.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:39:52.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA3DZRpuQcFORAA3q2IFhQgPCOLrCbSj9U2yw== --osd-uuid c328d2ca-818b-4cc0-b5a2-025d05e4b2e2 2026-02-17T06:39:52.074 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:52.068+0000 7fa65adad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:52.075 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:52.068+0000 7fa65adad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:52.076 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:52.068+0000 7fa65adad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:52.077 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:52.068+0000 7fa65adad8c0 -1 bdev(0x55906b183c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:39:52.077 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:52.068+0000 7fa65adad8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:39:55.138 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:39:55.139 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:39:55.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:39:55.140 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:39:55.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:39:55.336 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:39:55.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:39:55.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:39:55.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:39:55.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:39:55.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:39:55.351 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:55.344+0000 7f59d87498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:55.352 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:55.344+0000 7f59d87498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:55.353 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:55.348+0000 7f59d87498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:39:55.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:39:55.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:39:55.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:39:55.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:39:55.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:39:55.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:39:55.490 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:39:55.490 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:39:55.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:39:55.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:39:55.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:39:56.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:39:56.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:39:56.636 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:39:56.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:39:56.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:39:56.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:39:56.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:39:56.843 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:56.836+0000 7f59d87498c0 -1 Falling back to public interface 2026-02-17T06:39:57.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:39:57.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:39:57.788 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:39:57.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:39:57.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:39:57.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:39:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:39:58.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:39:58.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:39:58.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:39:58.937 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:39:58.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:39:58.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:39:59.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:39:59.805 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:39:59.796+0000 7f59d87498c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:40:00.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:00.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:00.086 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:40:00.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:40:00.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:00.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:00.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:01.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:01.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:01.233 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:40:01.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:40:01.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:01.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:01.382 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3823989411,v1:127.0.0.1:6803/3823989411] [v2:127.0.0.1:6804/3823989411,v1:127.0.0.1:6805/3823989411] exists,up c328d2ca-818b-4cc0-b5a2-025d05e4b2e2 2026-02-17T06:40:01.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:40:01.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:40:01.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:40:01.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 1 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:01.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:40:01.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:40:01.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:40:01.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b4dfce8c-b870-4a3e-b6e9-78160f64d324 2026-02-17T06:40:01.387 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 b4dfce8c-b870-4a3e-b6e9-78160f64d324 2026-02-17T06:40:01.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 b4dfce8c-b870-4a3e-b6e9-78160f64d324' 2026-02-17T06:40:01.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:40:01.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBBDZRpy2dyFxAA8uvXH821voJS2oD8w+a1Cw== 2026-02-17T06:40:01.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBBDZRpy2dyFxAA8uvXH821voJS2oD8w+a1Cw=="}' 2026-02-17T06:40:01.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b4dfce8c-b870-4a3e-b6e9-78160f64d324 -i td/osd-scrub-test/1/new.json 2026-02-17T06:40:01.550 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:40:01.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:40:01.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBDZRpy2dyFxAA8uvXH821voJS2oD8w+a1Cw== --osd-uuid b4dfce8c-b870-4a3e-b6e9-78160f64d324 2026-02-17T06:40:01.575 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:01.568+0000 7f2583c8c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:01.576 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:01.568+0000 7f2583c8c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:01.577 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:01.568+0000 7f2583c8c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:01.578 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:01.572+0000 7f2583c8c8c0 -1 bdev(0x55f8463a9c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:40:01.578 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:01.572+0000 7f2583c8c8c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:40:03.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:40:03.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:40:03.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:40:03.860 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:40:03.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:40:04.051 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:40:04.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:40:04.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:40:04.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:40:04.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:40:04.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:40:04.073 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:04.064+0000 7f7347d218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:04.074 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:04.068+0000 7f7347d218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:04.075 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:04.068+0000 7f7347d218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:04.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:40:04.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:40:04.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:40:04.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:40:04.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:40:04.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:04.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:40:04.205 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:40:04.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:04.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:04.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:05.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:05.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:05.358 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:40:05.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:40:05.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:05.359 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:05.504 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:05.567 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:05.560+0000 7f7347d218c0 -1 Falling back to public interface 2026-02-17T06:40:06.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:06.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:06.505 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:40:06.506 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:40:06.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:06.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:06.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:07.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:07.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:07.653 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:40:07.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:40:07.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:07.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:07.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:08.565 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:08.556+0000 7f7347d218c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:40:08.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:08.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:08.804 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:40:08.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:40:08.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:08.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:08.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:09.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:09.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:09.957 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:40:09.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:40:09.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:09.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:10.109 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3475086148,v1:127.0.0.1:6811/3475086148] [v2:127.0.0.1:6812/3475086148,v1:127.0.0.1:6813/3475086148] exists,up b4dfce8c-b870-4a3e-b6e9-78160f64d324 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 2 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:40:10.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:40:10.114 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:40:10.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:40:10.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:40:10.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:40:10.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:40:10.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:40:10.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:40:10.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:40:10.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=76e88864-d888-4210-89e8-bb65505b74f2 2026-02-17T06:40:10.117 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 76e88864-d888-4210-89e8-bb65505b74f2 2026-02-17T06:40:10.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 76e88864-d888-4210-89e8-bb65505b74f2' 2026-02-17T06:40:10.118 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:40:10.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBKDZRpw+8pBxAAVG4lBB7Ms7hFFGWVEv30kQ== 2026-02-17T06:40:10.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBKDZRpw+8pBxAAVG4lBB7Ms7hFFGWVEv30kQ=="}' 2026-02-17T06:40:10.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 76e88864-d888-4210-89e8-bb65505b74f2 -i td/osd-scrub-test/2/new.json 2026-02-17T06:40:10.273 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:40:10.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T06:40:10.283 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBKDZRpw+8pBxAAVG4lBB7Ms7hFFGWVEv30kQ== --osd-uuid 76e88864-d888-4210-89e8-bb65505b74f2 2026-02-17T06:40:10.298 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:10.292+0000 7fd3454638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:10.300 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:10.292+0000 7fd3454638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:10.302 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:10.292+0000 7fd3454638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:10.302 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:10.296+0000 7fd3454638c0 -1 bdev(0x56457329bc00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:40:10.303 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:10.296+0000 7fd3454638c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:40:12.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T06:40:12.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:40:12.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:40:12.844 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:40:12.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:40:13.032 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:40:13.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:40:13.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:40:13.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:40:13.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:40:13.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:40:13.054 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:13.048+0000 7fc52bd538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:13.056 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:13.048+0000 7fc52bd538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:13.057 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:13.048+0000 7fc52bd538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:13.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:40:13.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:40:13.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:40:13.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:40:13.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:40:13.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:13.186 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:40:13.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:40:13.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:13.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:13.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:14.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:14.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:14.331 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:40:14.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:40:14.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:14.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:14.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:14.543 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:14.536+0000 7fc52bd538c0 -1 Falling back to public interface 2026-02-17T06:40:15.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:15.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:15.478 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:40:15.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:40:15.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:15.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:15.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:16.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:16.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:16.628 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:40:16.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:40:16.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:16.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:16.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:17.537 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:17.528+0000 7fc52bd538c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:40:17.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:17.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:17.776 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:40:17.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:40:17.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:17.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:17.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:18.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:18.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:18.931 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:40:18.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:40:18.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:18.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:19.078 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/117960048,v1:127.0.0.1:6819/117960048] [v2:127.0.0.1:6820/117960048,v1:127.0.0.1:6821/117960048] exists,up 76e88864-d888-4210-89e8-bb65505b74f2 2026-02-17T06:40:19.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:40:19.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:40:19.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:40:19.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:668: TEST_pg_dump_objects_scrubbed: create_pool test 1 1 2026-02-17T06:40:19.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:40:19.274 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:40:19.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:40:20.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:669: TEST_pg_dump_objects_scrubbed: wait_for_clean 2026-02-17T06:40:20.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:40:20.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:40:20.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:40:20.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:40:20.288 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:40:20.288 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:40:20.288 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:40:20.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:40:20.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:40:20.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:40:20.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:40:20.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:40:20.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:40:20.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:40:20.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:40:20.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:40:20.549 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:40:20.549 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:40:20.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:40:20.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:20.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:40:20.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T06:40:20.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T06:40:20.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T06:40:20.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:20.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:40:20.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705676 2026-02-17T06:40:20.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705676 2026-02-17T06:40:20.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676' 2026-02-17T06:40:20.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:20.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:40:20.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-17T06:40:20.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-17T06:40:20.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676 2-55834574852' 2026-02-17T06:40:20.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:20.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T06:40:20.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:20.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:40:20.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T06:40:20.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:20.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T06:40:20.767 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836501 2026-02-17T06:40:20.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T06:40:20.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:40:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836501 2026-02-17T06:40:20.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:40:21.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:40:21.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:40:22.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836501 2026-02-17T06:40:22.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:40:23.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T06:40:23.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:40:23.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836503 -lt 21474836501 2026-02-17T06:40:23.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:23.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705676 2026-02-17T06:40:23.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:23.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:40:23.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705676 2026-02-17T06:40:23.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:23.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705676 2026-02-17T06:40:23.228 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705676 2026-02-17T06:40:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-17T06:40:23.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:40:23.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705678 -lt 38654705676 2026-02-17T06:40:23.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:23.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-17T06:40:23.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:23.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:40:23.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-17T06:40:23.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:23.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-17T06:40:23.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-17T06:40:23.392 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574852 2026-02-17T06:40:23.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:40:23.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574853 -lt 55834574852 2026-02-17T06:40:23.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:40:23.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:40:23.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:40:23.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:40:23.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:40:23.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:40:23.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:40:23.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:40:24.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:40:24.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:40:24.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:40:24.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: ceph osd dump 2026-02-17T06:40:24.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: grep '^pool.*['\'']test['\'']' 2026-02-17T06:40:24.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: awk '{ print $2 }' 2026-02-17T06:40:24.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: poolid=1 2026-02-17T06:40:24.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:672: TEST_pg_dump_objects_scrubbed: dd if=/dev/urandom of=testdata.507902 bs=1032 count=1 2026-02-17T06:40:24.215 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:40:24.215 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:40:24.215 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.342e-05 s, 19.3 MB/s 2026-02-17T06:40:24.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: seq 1 15 2026-02-17T06:40:24.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj1 testdata.507902 2026-02-17T06:40:24.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj2 testdata.507902 2026-02-17T06:40:24.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj3 testdata.507902 2026-02-17T06:40:24.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj4 testdata.507902 2026-02-17T06:40:24.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj5 testdata.507902 2026-02-17T06:40:24.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj6 testdata.507902 2026-02-17T06:40:24.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj7 testdata.507902 2026-02-17T06:40:24.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj8 testdata.507902 2026-02-17T06:40:24.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj9 testdata.507902 2026-02-17T06:40:24.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj10 testdata.507902 2026-02-17T06:40:24.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj11 testdata.507902 2026-02-17T06:40:24.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj12 testdata.507902 2026-02-17T06:40:24.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj13 testdata.507902 2026-02-17T06:40:24.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj14 testdata.507902 2026-02-17T06:40:24.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T06:40:24.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj15 testdata.507902 2026-02-17T06:40:24.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rm -f testdata.507902 2026-02-17T06:40:24.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:679: TEST_pg_dump_objects_scrubbed: local pgid=1.0 2026-02-17T06:40:24.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:681: TEST_pg_dump_objects_scrubbed: pg_scrub 1.0 2026-02-17T06:40:24.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T06:40:24.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T06:40:24.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T06:40:24.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:40:24.516 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:40:24.517 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:40:24.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:40:24.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:40:24.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:40:24.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:40:24.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:40:24.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:40:24.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:40:24.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:40:24.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:40:24.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:40:24.947 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:40:24.948 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:40:24.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:40:24.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:24.948 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:40:25.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836507 2026-02-17T06:40:25.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836507 2026-02-17T06:40:25.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507' 2026-02-17T06:40:25.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:25.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:40:25.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705682 2026-02-17T06:40:25.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705682 2026-02-17T06:40:25.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705682' 2026-02-17T06:40:25.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:25.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:40:25.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-17T06:40:25.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-17T06:40:25.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705682 2-55834574857' 2026-02-17T06:40:25.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:25.160 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836507 2026-02-17T06:40:25.160 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:25.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:40:25.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836507 2026-02-17T06:40:25.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:25.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836507 2026-02-17T06:40:25.163 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836507 2026-02-17T06:40:25.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-17T06:40:25.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:40:25.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836505 -lt 21474836507 2026-02-17T06:40:25.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:40:26.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:40:26.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:40:26.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836507 2026-02-17T06:40:26.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:26.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705682 2026-02-17T06:40:26.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:26.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:40:26.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705682 2026-02-17T06:40:26.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:26.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705682 2026-02-17T06:40:26.476 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705682 2026-02-17T06:40:26.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-17T06:40:26.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:40:26.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705683 -lt 38654705682 2026-02-17T06:40:26.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:26.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-17T06:40:26.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:26.629 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:40:26.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-17T06:40:26.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:26.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-17T06:40:26.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T06:40:26.632 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 55834574857 2026-02-17T06:40:26.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:40:26.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-17T06:40:26.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:40:26.790 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 1.0 loop 0 2026-02-17T06:40:26.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T06:40:26.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T06:40:26.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T06:40:26.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:40:26.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T06:40:26.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:40:26.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:40:26.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:40:26.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:40:26.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:40:26.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T06:40:26.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:40:26.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:40:26.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:40:26.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:40:27.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T06:40:19.268687+0000 2026-02-17T06:40:27.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T06:40:27.147 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-17T06:40:27.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T06:40:19.268687+0000 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:40:19.268687+0000 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:40:27.157 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:40:27.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:40:19.268687+0000 '>' 2026-02-17T06:40:19.268687+0000 2026-02-17T06:40:27.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:40:28.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:40:28.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:40:28.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:40:28.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:40:28.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:40:28.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:40:28.308 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:40:28.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:40:19.268687+0000 '>' 2026-02-17T06:40:19.268687+0000 2026-02-17T06:40:28.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:40:29.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:40:29.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:40:29.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:40:29.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:40:29.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:40:29.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:40:29.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:40:29.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:40:19.268687+0000 '>' 2026-02-17T06:40:19.268687+0000 2026-02-17T06:40:29.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:40:30.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:40:30.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:40:30.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:40:30.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:40:30.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:40:30.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:40:30.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:40:30.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:40:27.842555+0000 '>' 2026-02-17T06:40:19.268687+0000 2026-02-17T06:40:30.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:40:30.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:682: TEST_pg_dump_objects_scrubbed: ceph pg 1.0 query 2026-02-17T06:40:30.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:682: TEST_pg_dump_objects_scrubbed: jq .info.stats.objects_scrubbed 2026-02-17T06:40:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:682: TEST_pg_dump_objects_scrubbed: test 15 = 15 2026-02-17T06:40:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:683: TEST_pg_dump_objects_scrubbed: perf_counters td/osd-scrub-test 3 2026-02-17T06:40:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:40:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T06:40:30.834 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T06:40:30.834 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T06:40:30.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:30.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:40:30.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:40:30.908 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:40:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:40:30.908 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.909 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.910 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.911 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:30.912 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.913 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.914 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.915 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:30.916 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.917 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.918 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.919 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:30.920 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.921 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.922 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:30.923 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.923 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.923 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.923 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.923 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.924 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.925 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.925 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:30.925 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.925 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.925 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:40:30.925 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:40:30.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:30.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:40:30.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.983 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:40:30.984 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.986 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.987 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:40:30.988 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.989 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.990 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:30.991 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 1, 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 1, 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:30.992 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:30.993 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 1, 2026-02-17T06:40:30.994 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:30.994 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:30.994 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:30.994 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:30.994 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:30.994 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 1, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:30.995 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:30.996 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:30.996 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:40:30.996 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.996 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:30.996 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:40:30.996 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:40:30.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:30.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T06:40:30.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:31.054 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.055 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:40:31.056 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.057 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:31.058 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.059 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:40:31.060 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:40:31.061 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:40:31.062 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:40:31.063 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.063 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.063 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.063 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.064 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:40:31.065 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:40:31.065 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:40:31.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:685: TEST_pg_dump_objects_scrubbed: teardown td/osd-scrub-test 2026-02-17T06:40:31.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:40:31.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:40:31.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:40:31.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:40:31.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:40:31.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:40:31.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:40:31.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:40:31.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:40:31.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:40:31.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:40:31.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:40:31.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:40:31.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:40:31.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:40:31.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:40:31.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:40:31.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:40:31.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:40:31.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:40:31.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:40:31.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:40:31.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:40:31.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:40:31.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:40:31.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:40:31.184 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_pg_dump_objects_scrubbed ------------------ 2026-02-17T06:40:31.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_pg_dump_objects_scrubbed ------------------' 2026-02-17T06:40:31.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:40:31.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:40:31.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:40:31.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:40:31.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:40:31.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:40:31.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:40:31.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:40:31.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:40:31.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:40:31.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:40:31.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:40:31.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:40:31.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:40:31.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:40:31.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:40:31.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:40:31.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:40:31.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:40:31.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:40:31.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:40:31.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:40:31.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:40:31.194 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:40:31.194 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.194 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:40:31.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:40:31.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:40:31.196 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_pg_dump_objects_scrubbed ------------------ 2026-02-17T06:40:31.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_pg_dump_objects_scrubbed ------------------' 2026-02-17T06:40:31.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:40:31.197 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_scrub_abort ------------------- 2026-02-17T06:40:31.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_abort -------------------' 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:40:31.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:40:31.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:40:31.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:40:31.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:40:31.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:40:31.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:40:31.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:40:31.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:40:31.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:40:31.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:40:31.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:40:31.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:40:31.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:40:31.204 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:40:31.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:40:31.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:40:31.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:40:31.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:40:31.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.208 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:40:31.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:40:31.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:40:31.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:40:31.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:40:31.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:40:31.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:40:31.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:40:31.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:40:31.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:40:31.214 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_scrub_abort ----------------------- 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_abort -----------------------' 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_abort td/osd-scrub-test 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:409: TEST_scrub_abort: local dir=td/osd-scrub-test 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:410: TEST_scrub_abort: _scrub_abort td/osd-scrub-test scrub 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:298: _scrub_abort: local dir=td/osd-scrub-test 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:299: _scrub_abort: local poolname=test 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:300: _scrub_abort: local OSDS=3 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:301: _scrub_abort: local objects=1000 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:302: _scrub_abort: local type=scrub 2026-02-17T06:40:31.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:304: _scrub_abort: TESTDATA=testdata.507902 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:305: _scrub_abort: test scrub = scrub 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:307: _scrub_abort: stopscrub=noscrub 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:308: _scrub_abort: check=noscrub 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:314: _scrub_abort: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:40:31.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T06:40:31.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:40:31.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:40:31.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:40:31.235 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:40:31.235 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.235 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:31.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T06:40:31.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:40:31.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:40:31.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:40:31.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:40:31.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:40:31.265 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:40:31.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:40:31.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:40:31.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:40:31.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:40:31.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:40:31.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:40:31.332 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:40:31.332 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:40:31.332 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:40:31.333 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:40:31.333 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.333 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.333 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:40:31.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:40:31.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:40:31.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:315: _scrub_abort: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:40:31.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:40:31.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:40:31.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:40:31.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:40:31.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:40:31.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:40:31.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:40:31.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:40:31.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:40:31.483 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:40:31.483 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.483 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:31.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:40:31.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:40:31.502 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: expr 3 - 1 2026-02-17T06:40:31.503 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: seq 0 2 2026-02-17T06:40:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 0 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:40:31.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:40:31.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:40:31.509 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:40:31.510 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:40:31.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=00362a94-a6f6-4186-9df0-19dca6966ceb 2026-02-17T06:40:31.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 00362a94-a6f6-4186-9df0-19dca6966ceb' 2026-02-17T06:40:31.511 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 00362a94-a6f6-4186-9df0-19dca6966ceb 2026-02-17T06:40:31.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:40:31.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBfDZRpmzmWHhAAo+A0oWhUgbarT/3qeS3UXQ== 2026-02-17T06:40:31.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBfDZRpmzmWHhAAo+A0oWhUgbarT/3qeS3UXQ=="}' 2026-02-17T06:40:31.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 00362a94-a6f6-4186-9df0-19dca6966ceb -i td/osd-scrub-test/0/new.json 2026-02-17T06:40:31.607 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:40:31.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:40:31.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBfDZRpmzmWHhAAo+A0oWhUgbarT/3qeS3UXQ== --osd-uuid 00362a94-a6f6-4186-9df0-19dca6966ceb 2026-02-17T06:40:31.632 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:31.623+0000 7f6166d398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:31.633 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:31.623+0000 7f6166d398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:31.634 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:31.627+0000 7f6166d398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:31.635 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:31.627+0000 7f6166d398c0 -1 bdev(0x55cf954e5c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:40:31.635 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:31.627+0000 7f6166d398c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:40:34.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:40:34.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:40:34.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:40:34.448 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:40:34.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:40:34.642 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:40:34.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:40:34.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:40:34.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:40:34.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:40:34.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:40:34.665 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:34.655+0000 7f1b2ca6f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:34.667 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:34.659+0000 7f1b2ca6f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:34.668 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:34.659+0000 7f1b2ca6f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:34.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:40:34.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:40:34.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:40:34.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:40:34.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:40:34.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:34.795 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:40:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:40:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:34.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:34.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:35.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:35.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:35.952 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:40:35.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:40:35.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:35.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:36.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:36.368 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:36.359+0000 7f1b2ca6f8c0 -1 Falling back to public interface 2026-02-17T06:40:37.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:37.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:37.105 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:40:37.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:40:37.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:37.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:37.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:38.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:38.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:38.255 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:40:38.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:40:38.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:38.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:38.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:39.351 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:39.343+0000 7f1b2ca6f8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:40:39.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:39.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:39.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:40:39.412 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:40:39.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:39.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:39.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:40.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:40.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:40.568 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:40:40.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:40:40.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:40.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:40:40.719 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4165866385,v1:127.0.0.1:6803/4165866385] [v2:127.0.0.1:6804/4165866385,v1:127.0.0.1:6805/4165866385] exists,up 00362a94-a6f6-4186-9df0-19dca6966ceb 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 1 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:40:40.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:40:40.723 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:40:40.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:40:40.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:40:40.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:40:40.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:40:40.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:40:40.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T06:40:40.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:40:40.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:40:40.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a92d68a2-3348-4f12-bcc5-d9c3d095a68a 2026-02-17T06:40:40.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 a92d68a2-3348-4f12-bcc5-d9c3d095a68a' 2026-02-17T06:40:40.726 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 a92d68a2-3348-4f12-bcc5-d9c3d095a68a 2026-02-17T06:40:40.726 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:40:40.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBoDZRpYbKJKxAAvY9eQ735NZeXAesiSuv4hg== 2026-02-17T06:40:40.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBoDZRpYbKJKxAAvY9eQ735NZeXAesiSuv4hg=="}' 2026-02-17T06:40:40.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a92d68a2-3348-4f12-bcc5-d9c3d095a68a -i td/osd-scrub-test/1/new.json 2026-02-17T06:40:40.893 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:40:40.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:40:40.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBoDZRpYbKJKxAAvY9eQ735NZeXAesiSuv4hg== --osd-uuid a92d68a2-3348-4f12-bcc5-d9c3d095a68a 2026-02-17T06:40:40.919 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:40.911+0000 7ff182cf28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:40.921 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:40.911+0000 7ff182cf28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:40.922 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:40.915+0000 7ff182cf28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:40.923 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:40.915+0000 7ff182cf28c0 -1 bdev(0x560e46d3bc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:40:40.923 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:40.915+0000 7ff182cf28c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:40:43.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:40:43.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:40:43.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:40:43.224 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:40:43.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:40:43.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:40:43.416 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:40:43.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:40:43.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:40:43.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:40:43.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:40:43.439 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:43.431+0000 7fd65570f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:43.441 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:43.431+0000 7fd65570f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:43.442 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:43.435+0000 7fd65570f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:43.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:40:43.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:40:43.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:40:43.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:40:43.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:40:43.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:43.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:40:43.567 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:40:43.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:43.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:43.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:44.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:44.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:44.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:40:44.723 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:40:44.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:44.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:44.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:44.903 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:44.895+0000 7fd65570f8c0 -1 Falling back to public interface 2026-02-17T06:40:45.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:45.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:45.874 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:40:45.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:40:45.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:45.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:46.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:47.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:47.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:47.025 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:40:47.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:40:47.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:47.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:47.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:47.877 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:47.867+0000 7fd65570f8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:40:48.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:48.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:48.178 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:40:48.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:40:48.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:48.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:40:48.329 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2697412745,v1:127.0.0.1:6811/2697412745] [v2:127.0.0.1:6812/2697412745,v1:127.0.0.1:6813/2697412745] exists,up a92d68a2-3348-4f12-bcc5-d9c3d095a68a 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 2 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:40:48.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:40:48.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:40:48.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:40:48.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=875f8b99-763a-427d-bc19-e052897a25dc 2026-02-17T06:40:48.338 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 875f8b99-763a-427d-bc19-e052897a25dc 2026-02-17T06:40:48.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 875f8b99-763a-427d-bc19-e052897a25dc' 2026-02-17T06:40:48.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:40:48.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBwDZRpo2E2FBAApL1SOUkisejoXuPMkREteQ== 2026-02-17T06:40:48.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBwDZRpo2E2FBAApL1SOUkisejoXuPMkREteQ=="}' 2026-02-17T06:40:48.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 875f8b99-763a-427d-bc19-e052897a25dc -i td/osd-scrub-test/2/new.json 2026-02-17T06:40:48.491 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:40:48.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T06:40:48.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQBwDZRpo2E2FBAApL1SOUkisejoXuPMkREteQ== --osd-uuid 875f8b99-763a-427d-bc19-e052897a25dc 2026-02-17T06:40:48.527 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:48.519+0000 7f8249e648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:48.528 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:48.519+0000 7f8249e648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:48.530 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:48.523+0000 7f8249e648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:48.530 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:48.523+0000 7f8249e648c0 -1 bdev(0x556f03759c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:40:48.530 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:48.523+0000 7f8249e648c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:40:50.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T06:40:50.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:40:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:40:50.828 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:40:50.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:40:51.024 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:40:51.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:40:51.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T06:40:51.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:40:51.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:40:51.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:40:51.049 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:51.039+0000 7fcc37ca98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:51.050 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:51.043+0000 7fcc37ca98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:51.051 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:51.043+0000 7fcc37ca98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:40:51.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:40:51.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:40:51.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:40:51.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:40:51.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:40:51.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:51.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:40:51.173 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:40:51.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:51.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:51.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:52.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:52.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:52.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:40:52.327 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:40:52.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:52.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:52.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:52.527 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:52.519+0000 7fcc37ca98c0 -1 Falling back to public interface 2026-02-17T06:40:53.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:53.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:53.479 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:40:53.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:40:53.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:53.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:53.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:54.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:54.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:54.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:40:54.634 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:40:54.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:54.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:54.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:40:55.574 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:40:55.567+0000 7fcc37ca98c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:40:55.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:40:55.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:40:55.785 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:40:55.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:40:55.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:40:55.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:40:55.936 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3389219778,v1:127.0.0.1:6819/3389219778] [v2:127.0.0.1:6820/3389219778,v1:127.0.0.1:6821/3389219778] exists,up 875f8b99-763a-427d-bc19-e052897a25dc 2026-02-17T06:40:55.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:40:55.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:40:55.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:40:55.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: create_pool test 1 1 2026-02-17T06:40:55.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:40:56.138 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:40:56.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:40:57.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:331: _scrub_abort: wait_for_clean 2026-02-17T06:40:57.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:40:57.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:40:57.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:40:57.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:40:57.153 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:40:57.153 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:40:57.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:40:57.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:40:57.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:40:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:40:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:40:57.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:40:57.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:40:57.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:40:57.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:40:57.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:40:57.411 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:40:57.411 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:40:57.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:40:57.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:57.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:40:57.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T06:40:57.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T06:40:57.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T06:40:57.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:57.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:40:57.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T06:40:57.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T06:40:57.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705674' 2026-02-17T06:40:57.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:40:57.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:40:57.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T06:40:57.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T06:40:57.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705674 2-60129542146' 2026-02-17T06:40:57.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:57.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T06:40:57.628 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:57.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:40:57.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T06:40:57.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:57.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T06:40:57.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T06:40:57.631 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836498 2026-02-17T06:40:57.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:40:57.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836498 2026-02-17T06:40:57.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:40:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:40:58.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:40:58.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836498 2026-02-17T06:40:58.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:58.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T06:40:58.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:58.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:40:58.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T06:40:58.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:58.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T06:40:58.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T06:40:58.936 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705674 2026-02-17T06:40:58.936 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:40:59.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T06:40:59.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:40:59.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T06:40:59.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:40:59.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:40:59.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T06:40:59.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:40:59.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T06:40:59.089 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542146 2026-02-17T06:40:59.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T06:40:59.089 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:40:59.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542146 2026-02-17T06:40:59.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:40:59.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:40:59.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:40:59.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:40:59.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:40:59.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:40:59.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:40:59.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:40:59.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:40:59.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:40:59.418 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:40:59.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:40:59.563 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:40:59.563 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:40:59.563 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:40:59.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:40:59.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:40:59.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:40:59.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: ceph osd dump 2026-02-17T06:40:59.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: grep '^pool.*['\'']test['\'']' 2026-02-17T06:40:59.740 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: awk '{ print $2 }' 2026-02-17T06:40:59.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: poolid=1 2026-02-17T06:40:59.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:334: _scrub_abort: dd if=/dev/urandom of=testdata.507902 bs=1032 count=1 2026-02-17T06:40:59.888 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:40:59.888 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:40:59.888 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.3834e-05 s, 30.5 MB/s 2026-02-17T06:40:59.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: seq 1 1000 2026-02-17T06:40:59.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:40:59.889 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1 testdata.507902 2026-02-17T06:40:59.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:40:59.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj2 testdata.507902 2026-02-17T06:40:59.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:40:59.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj3 testdata.507902 2026-02-17T06:40:59.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:40:59.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj4 testdata.507902 2026-02-17T06:40:59.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:40:59.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj5 testdata.507902 2026-02-17T06:41:00.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj6 testdata.507902 2026-02-17T06:41:00.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj7 testdata.507902 2026-02-17T06:41:00.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj8 testdata.507902 2026-02-17T06:41:00.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj9 testdata.507902 2026-02-17T06:41:00.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj10 testdata.507902 2026-02-17T06:41:00.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj11 testdata.507902 2026-02-17T06:41:00.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj12 testdata.507902 2026-02-17T06:41:00.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj13 testdata.507902 2026-02-17T06:41:00.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj14 testdata.507902 2026-02-17T06:41:00.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj15 testdata.507902 2026-02-17T06:41:00.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj16 testdata.507902 2026-02-17T06:41:00.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj17 testdata.507902 2026-02-17T06:41:00.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj18 testdata.507902 2026-02-17T06:41:00.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj19 testdata.507902 2026-02-17T06:41:00.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj20 testdata.507902 2026-02-17T06:41:00.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj21 testdata.507902 2026-02-17T06:41:00.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj22 testdata.507902 2026-02-17T06:41:00.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj23 testdata.507902 2026-02-17T06:41:00.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj24 testdata.507902 2026-02-17T06:41:00.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj25 testdata.507902 2026-02-17T06:41:00.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj26 testdata.507902 2026-02-17T06:41:00.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj27 testdata.507902 2026-02-17T06:41:00.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj28 testdata.507902 2026-02-17T06:41:00.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj29 testdata.507902 2026-02-17T06:41:00.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj30 testdata.507902 2026-02-17T06:41:00.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj31 testdata.507902 2026-02-17T06:41:00.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj32 testdata.507902 2026-02-17T06:41:00.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj33 testdata.507902 2026-02-17T06:41:00.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj34 testdata.507902 2026-02-17T06:41:00.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj35 testdata.507902 2026-02-17T06:41:00.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj36 testdata.507902 2026-02-17T06:41:00.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj37 testdata.507902 2026-02-17T06:41:00.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj38 testdata.507902 2026-02-17T06:41:00.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj39 testdata.507902 2026-02-17T06:41:00.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj40 testdata.507902 2026-02-17T06:41:00.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj41 testdata.507902 2026-02-17T06:41:00.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj42 testdata.507902 2026-02-17T06:41:00.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj43 testdata.507902 2026-02-17T06:41:00.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj44 testdata.507902 2026-02-17T06:41:00.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj45 testdata.507902 2026-02-17T06:41:00.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj46 testdata.507902 2026-02-17T06:41:00.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj47 testdata.507902 2026-02-17T06:41:00.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj48 testdata.507902 2026-02-17T06:41:00.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:00.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj49 testdata.507902 2026-02-17T06:41:01.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj50 testdata.507902 2026-02-17T06:41:01.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj51 testdata.507902 2026-02-17T06:41:01.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj52 testdata.507902 2026-02-17T06:41:01.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj53 testdata.507902 2026-02-17T06:41:01.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj54 testdata.507902 2026-02-17T06:41:01.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj55 testdata.507902 2026-02-17T06:41:01.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj56 testdata.507902 2026-02-17T06:41:01.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj57 testdata.507902 2026-02-17T06:41:01.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj58 testdata.507902 2026-02-17T06:41:01.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj59 testdata.507902 2026-02-17T06:41:01.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj60 testdata.507902 2026-02-17T06:41:01.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj61 testdata.507902 2026-02-17T06:41:01.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj62 testdata.507902 2026-02-17T06:41:01.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj63 testdata.507902 2026-02-17T06:41:01.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj64 testdata.507902 2026-02-17T06:41:01.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj65 testdata.507902 2026-02-17T06:41:01.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj66 testdata.507902 2026-02-17T06:41:01.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj67 testdata.507902 2026-02-17T06:41:01.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj68 testdata.507902 2026-02-17T06:41:01.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj69 testdata.507902 2026-02-17T06:41:01.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj70 testdata.507902 2026-02-17T06:41:01.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj71 testdata.507902 2026-02-17T06:41:01.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj72 testdata.507902 2026-02-17T06:41:01.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj73 testdata.507902 2026-02-17T06:41:01.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj74 testdata.507902 2026-02-17T06:41:01.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj75 testdata.507902 2026-02-17T06:41:01.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj76 testdata.507902 2026-02-17T06:41:01.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj77 testdata.507902 2026-02-17T06:41:01.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj78 testdata.507902 2026-02-17T06:41:01.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj79 testdata.507902 2026-02-17T06:41:01.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj80 testdata.507902 2026-02-17T06:41:01.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj81 testdata.507902 2026-02-17T06:41:01.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj82 testdata.507902 2026-02-17T06:41:01.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj83 testdata.507902 2026-02-17T06:41:01.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj84 testdata.507902 2026-02-17T06:41:01.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj85 testdata.507902 2026-02-17T06:41:01.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj86 testdata.507902 2026-02-17T06:41:01.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj87 testdata.507902 2026-02-17T06:41:01.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj88 testdata.507902 2026-02-17T06:41:01.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj89 testdata.507902 2026-02-17T06:41:01.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj90 testdata.507902 2026-02-17T06:41:01.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj91 testdata.507902 2026-02-17T06:41:01.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:01.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj92 testdata.507902 2026-02-17T06:41:02.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj93 testdata.507902 2026-02-17T06:41:02.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj94 testdata.507902 2026-02-17T06:41:02.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj95 testdata.507902 2026-02-17T06:41:02.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj96 testdata.507902 2026-02-17T06:41:02.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj97 testdata.507902 2026-02-17T06:41:02.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj98 testdata.507902 2026-02-17T06:41:02.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj99 testdata.507902 2026-02-17T06:41:02.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj100 testdata.507902 2026-02-17T06:41:02.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj101 testdata.507902 2026-02-17T06:41:02.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj102 testdata.507902 2026-02-17T06:41:02.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj103 testdata.507902 2026-02-17T06:41:02.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.285 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj104 testdata.507902 2026-02-17T06:41:02.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj105 testdata.507902 2026-02-17T06:41:02.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.324 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj106 testdata.507902 2026-02-17T06:41:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj107 testdata.507902 2026-02-17T06:41:02.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj108 testdata.507902 2026-02-17T06:41:02.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj109 testdata.507902 2026-02-17T06:41:02.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj110 testdata.507902 2026-02-17T06:41:02.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.445 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj111 testdata.507902 2026-02-17T06:41:02.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj112 testdata.507902 2026-02-17T06:41:02.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj113 testdata.507902 2026-02-17T06:41:02.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj114 testdata.507902 2026-02-17T06:41:02.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj115 testdata.507902 2026-02-17T06:41:02.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj116 testdata.507902 2026-02-17T06:41:02.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj117 testdata.507902 2026-02-17T06:41:02.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj118 testdata.507902 2026-02-17T06:41:02.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj119 testdata.507902 2026-02-17T06:41:02.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj120 testdata.507902 2026-02-17T06:41:02.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj121 testdata.507902 2026-02-17T06:41:02.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj122 testdata.507902 2026-02-17T06:41:02.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj123 testdata.507902 2026-02-17T06:41:02.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj124 testdata.507902 2026-02-17T06:41:02.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj125 testdata.507902 2026-02-17T06:41:02.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj126 testdata.507902 2026-02-17T06:41:02.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj127 testdata.507902 2026-02-17T06:41:02.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj128 testdata.507902 2026-02-17T06:41:02.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj129 testdata.507902 2026-02-17T06:41:02.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj130 testdata.507902 2026-02-17T06:41:02.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj131 testdata.507902 2026-02-17T06:41:02.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj132 testdata.507902 2026-02-17T06:41:02.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj133 testdata.507902 2026-02-17T06:41:02.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:02.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj134 testdata.507902 2026-02-17T06:41:03.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj135 testdata.507902 2026-02-17T06:41:03.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.032 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj136 testdata.507902 2026-02-17T06:41:03.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj137 testdata.507902 2026-02-17T06:41:03.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj138 testdata.507902 2026-02-17T06:41:03.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj139 testdata.507902 2026-02-17T06:41:03.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj140 testdata.507902 2026-02-17T06:41:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj141 testdata.507902 2026-02-17T06:41:03.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj142 testdata.507902 2026-02-17T06:41:03.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj143 testdata.507902 2026-02-17T06:41:03.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj144 testdata.507902 2026-02-17T06:41:03.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj145 testdata.507902 2026-02-17T06:41:03.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj146 testdata.507902 2026-02-17T06:41:03.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj147 testdata.507902 2026-02-17T06:41:03.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj148 testdata.507902 2026-02-17T06:41:03.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj149 testdata.507902 2026-02-17T06:41:03.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.336 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj150 testdata.507902 2026-02-17T06:41:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj151 testdata.507902 2026-02-17T06:41:03.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj152 testdata.507902 2026-02-17T06:41:03.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj153 testdata.507902 2026-02-17T06:41:03.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj154 testdata.507902 2026-02-17T06:41:03.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj155 testdata.507902 2026-02-17T06:41:03.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj156 testdata.507902 2026-02-17T06:41:03.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj157 testdata.507902 2026-02-17T06:41:03.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj158 testdata.507902 2026-02-17T06:41:03.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj159 testdata.507902 2026-02-17T06:41:03.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj160 testdata.507902 2026-02-17T06:41:03.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj161 testdata.507902 2026-02-17T06:41:03.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj162 testdata.507902 2026-02-17T06:41:03.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj163 testdata.507902 2026-02-17T06:41:03.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj164 testdata.507902 2026-02-17T06:41:03.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj165 testdata.507902 2026-02-17T06:41:03.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj166 testdata.507902 2026-02-17T06:41:03.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.740 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj167 testdata.507902 2026-02-17T06:41:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj168 testdata.507902 2026-02-17T06:41:03.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj169 testdata.507902 2026-02-17T06:41:03.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj170 testdata.507902 2026-02-17T06:41:03.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj171 testdata.507902 2026-02-17T06:41:03.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj172 testdata.507902 2026-02-17T06:41:03.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj173 testdata.507902 2026-02-17T06:41:03.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj174 testdata.507902 2026-02-17T06:41:03.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj175 testdata.507902 2026-02-17T06:41:03.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj176 testdata.507902 2026-02-17T06:41:03.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj177 testdata.507902 2026-02-17T06:41:03.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:03.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj178 testdata.507902 2026-02-17T06:41:04.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj179 testdata.507902 2026-02-17T06:41:04.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj180 testdata.507902 2026-02-17T06:41:04.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj181 testdata.507902 2026-02-17T06:41:04.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj182 testdata.507902 2026-02-17T06:41:04.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj183 testdata.507902 2026-02-17T06:41:04.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj184 testdata.507902 2026-02-17T06:41:04.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj185 testdata.507902 2026-02-17T06:41:04.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj186 testdata.507902 2026-02-17T06:41:04.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj187 testdata.507902 2026-02-17T06:41:04.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj188 testdata.507902 2026-02-17T06:41:04.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj189 testdata.507902 2026-02-17T06:41:04.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj190 testdata.507902 2026-02-17T06:41:04.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj191 testdata.507902 2026-02-17T06:41:04.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj192 testdata.507902 2026-02-17T06:41:04.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj193 testdata.507902 2026-02-17T06:41:04.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj194 testdata.507902 2026-02-17T06:41:04.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj195 testdata.507902 2026-02-17T06:41:04.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj196 testdata.507902 2026-02-17T06:41:04.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj197 testdata.507902 2026-02-17T06:41:04.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj198 testdata.507902 2026-02-17T06:41:04.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj199 testdata.507902 2026-02-17T06:41:04.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj200 testdata.507902 2026-02-17T06:41:04.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj201 testdata.507902 2026-02-17T06:41:04.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj202 testdata.507902 2026-02-17T06:41:04.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj203 testdata.507902 2026-02-17T06:41:04.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj204 testdata.507902 2026-02-17T06:41:04.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj205 testdata.507902 2026-02-17T06:41:04.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj206 testdata.507902 2026-02-17T06:41:04.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj207 testdata.507902 2026-02-17T06:41:04.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj208 testdata.507902 2026-02-17T06:41:04.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj209 testdata.507902 2026-02-17T06:41:04.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj210 testdata.507902 2026-02-17T06:41:04.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj211 testdata.507902 2026-02-17T06:41:04.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj212 testdata.507902 2026-02-17T06:41:04.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj213 testdata.507902 2026-02-17T06:41:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj214 testdata.507902 2026-02-17T06:41:04.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj215 testdata.507902 2026-02-17T06:41:04.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj216 testdata.507902 2026-02-17T06:41:04.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj217 testdata.507902 2026-02-17T06:41:04.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj218 testdata.507902 2026-02-17T06:41:04.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj219 testdata.507902 2026-02-17T06:41:04.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj220 testdata.507902 2026-02-17T06:41:04.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj221 testdata.507902 2026-02-17T06:41:04.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:04.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj222 testdata.507902 2026-02-17T06:41:05.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj223 testdata.507902 2026-02-17T06:41:05.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj224 testdata.507902 2026-02-17T06:41:05.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj225 testdata.507902 2026-02-17T06:41:05.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj226 testdata.507902 2026-02-17T06:41:05.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj227 testdata.507902 2026-02-17T06:41:05.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj228 testdata.507902 2026-02-17T06:41:05.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.129 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj229 testdata.507902 2026-02-17T06:41:05.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj230 testdata.507902 2026-02-17T06:41:05.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj231 testdata.507902 2026-02-17T06:41:05.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj232 testdata.507902 2026-02-17T06:41:05.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj233 testdata.507902 2026-02-17T06:41:05.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj234 testdata.507902 2026-02-17T06:41:05.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj235 testdata.507902 2026-02-17T06:41:05.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj236 testdata.507902 2026-02-17T06:41:05.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.323 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj237 testdata.507902 2026-02-17T06:41:05.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj238 testdata.507902 2026-02-17T06:41:05.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj239 testdata.507902 2026-02-17T06:41:05.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj240 testdata.507902 2026-02-17T06:41:05.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj241 testdata.507902 2026-02-17T06:41:05.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj242 testdata.507902 2026-02-17T06:41:05.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj243 testdata.507902 2026-02-17T06:41:05.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj244 testdata.507902 2026-02-17T06:41:05.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj245 testdata.507902 2026-02-17T06:41:05.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj246 testdata.507902 2026-02-17T06:41:05.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj247 testdata.507902 2026-02-17T06:41:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj248 testdata.507902 2026-02-17T06:41:05.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj249 testdata.507902 2026-02-17T06:41:05.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj250 testdata.507902 2026-02-17T06:41:05.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj251 testdata.507902 2026-02-17T06:41:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj252 testdata.507902 2026-02-17T06:41:05.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj253 testdata.507902 2026-02-17T06:41:05.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj254 testdata.507902 2026-02-17T06:41:05.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.736 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj255 testdata.507902 2026-02-17T06:41:05.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj256 testdata.507902 2026-02-17T06:41:05.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj257 testdata.507902 2026-02-17T06:41:05.809 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj258 testdata.507902 2026-02-17T06:41:05.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj259 testdata.507902 2026-02-17T06:41:05.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj260 testdata.507902 2026-02-17T06:41:05.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj261 testdata.507902 2026-02-17T06:41:05.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj262 testdata.507902 2026-02-17T06:41:05.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj263 testdata.507902 2026-02-17T06:41:05.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj264 testdata.507902 2026-02-17T06:41:05.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj265 testdata.507902 2026-02-17T06:41:05.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:05.989 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj266 testdata.507902 2026-02-17T06:41:06.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj267 testdata.507902 2026-02-17T06:41:06.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.038 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj268 testdata.507902 2026-02-17T06:41:06.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj269 testdata.507902 2026-02-17T06:41:06.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj270 testdata.507902 2026-02-17T06:41:06.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj271 testdata.507902 2026-02-17T06:41:06.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj272 testdata.507902 2026-02-17T06:41:06.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj273 testdata.507902 2026-02-17T06:41:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj274 testdata.507902 2026-02-17T06:41:06.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj275 testdata.507902 2026-02-17T06:41:06.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj276 testdata.507902 2026-02-17T06:41:06.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj277 testdata.507902 2026-02-17T06:41:06.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj278 testdata.507902 2026-02-17T06:41:06.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj279 testdata.507902 2026-02-17T06:41:06.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj280 testdata.507902 2026-02-17T06:41:06.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj281 testdata.507902 2026-02-17T06:41:06.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj282 testdata.507902 2026-02-17T06:41:06.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj283 testdata.507902 2026-02-17T06:41:06.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj284 testdata.507902 2026-02-17T06:41:06.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj285 testdata.507902 2026-02-17T06:41:06.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj286 testdata.507902 2026-02-17T06:41:06.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj287 testdata.507902 2026-02-17T06:41:06.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj288 testdata.507902 2026-02-17T06:41:06.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj289 testdata.507902 2026-02-17T06:41:06.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj290 testdata.507902 2026-02-17T06:41:06.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj291 testdata.507902 2026-02-17T06:41:06.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj292 testdata.507902 2026-02-17T06:41:06.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj293 testdata.507902 2026-02-17T06:41:06.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj294 testdata.507902 2026-02-17T06:41:06.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj295 testdata.507902 2026-02-17T06:41:06.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj296 testdata.507902 2026-02-17T06:41:06.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj297 testdata.507902 2026-02-17T06:41:06.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj298 testdata.507902 2026-02-17T06:41:06.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj299 testdata.507902 2026-02-17T06:41:06.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj300 testdata.507902 2026-02-17T06:41:06.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj301 testdata.507902 2026-02-17T06:41:06.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj302 testdata.507902 2026-02-17T06:41:06.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj303 testdata.507902 2026-02-17T06:41:06.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj304 testdata.507902 2026-02-17T06:41:06.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj305 testdata.507902 2026-02-17T06:41:06.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj306 testdata.507902 2026-02-17T06:41:06.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj307 testdata.507902 2026-02-17T06:41:06.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:06.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj308 testdata.507902 2026-02-17T06:41:07.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj309 testdata.507902 2026-02-17T06:41:07.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj310 testdata.507902 2026-02-17T06:41:07.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj311 testdata.507902 2026-02-17T06:41:07.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj312 testdata.507902 2026-02-17T06:41:07.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj313 testdata.507902 2026-02-17T06:41:07.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj314 testdata.507902 2026-02-17T06:41:07.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj315 testdata.507902 2026-02-17T06:41:07.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj316 testdata.507902 2026-02-17T06:41:07.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj317 testdata.507902 2026-02-17T06:41:07.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj318 testdata.507902 2026-02-17T06:41:07.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj319 testdata.507902 2026-02-17T06:41:07.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj320 testdata.507902 2026-02-17T06:41:07.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj321 testdata.507902 2026-02-17T06:41:07.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj322 testdata.507902 2026-02-17T06:41:07.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj323 testdata.507902 2026-02-17T06:41:07.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj324 testdata.507902 2026-02-17T06:41:07.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj325 testdata.507902 2026-02-17T06:41:07.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj326 testdata.507902 2026-02-17T06:41:07.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj327 testdata.507902 2026-02-17T06:41:07.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.438 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj328 testdata.507902 2026-02-17T06:41:07.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj329 testdata.507902 2026-02-17T06:41:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj330 testdata.507902 2026-02-17T06:41:07.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj331 testdata.507902 2026-02-17T06:41:07.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj332 testdata.507902 2026-02-17T06:41:07.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj333 testdata.507902 2026-02-17T06:41:07.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj334 testdata.507902 2026-02-17T06:41:07.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj335 testdata.507902 2026-02-17T06:41:07.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj336 testdata.507902 2026-02-17T06:41:07.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj337 testdata.507902 2026-02-17T06:41:07.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj338 testdata.507902 2026-02-17T06:41:07.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj339 testdata.507902 2026-02-17T06:41:07.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj340 testdata.507902 2026-02-17T06:41:07.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj341 testdata.507902 2026-02-17T06:41:07.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj342 testdata.507902 2026-02-17T06:41:07.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj343 testdata.507902 2026-02-17T06:41:07.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj344 testdata.507902 2026-02-17T06:41:07.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj345 testdata.507902 2026-02-17T06:41:07.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj346 testdata.507902 2026-02-17T06:41:07.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj347 testdata.507902 2026-02-17T06:41:07.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj348 testdata.507902 2026-02-17T06:41:07.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj349 testdata.507902 2026-02-17T06:41:07.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj350 testdata.507902 2026-02-17T06:41:07.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj351 testdata.507902 2026-02-17T06:41:07.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj352 testdata.507902 2026-02-17T06:41:07.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:07.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj353 testdata.507902 2026-02-17T06:41:08.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj354 testdata.507902 2026-02-17T06:41:08.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj355 testdata.507902 2026-02-17T06:41:08.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj356 testdata.507902 2026-02-17T06:41:08.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj357 testdata.507902 2026-02-17T06:41:08.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj358 testdata.507902 2026-02-17T06:41:08.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj359 testdata.507902 2026-02-17T06:41:08.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj360 testdata.507902 2026-02-17T06:41:08.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj361 testdata.507902 2026-02-17T06:41:08.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj362 testdata.507902 2026-02-17T06:41:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj363 testdata.507902 2026-02-17T06:41:08.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj364 testdata.507902 2026-02-17T06:41:08.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj365 testdata.507902 2026-02-17T06:41:08.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj366 testdata.507902 2026-02-17T06:41:08.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj367 testdata.507902 2026-02-17T06:41:08.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj368 testdata.507902 2026-02-17T06:41:08.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj369 testdata.507902 2026-02-17T06:41:08.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj370 testdata.507902 2026-02-17T06:41:08.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj371 testdata.507902 2026-02-17T06:41:08.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj372 testdata.507902 2026-02-17T06:41:08.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.427 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj373 testdata.507902 2026-02-17T06:41:08.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj374 testdata.507902 2026-02-17T06:41:08.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj375 testdata.507902 2026-02-17T06:41:08.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.499 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj376 testdata.507902 2026-02-17T06:41:08.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj377 testdata.507902 2026-02-17T06:41:08.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj378 testdata.507902 2026-02-17T06:41:08.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj379 testdata.507902 2026-02-17T06:41:08.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj380 testdata.507902 2026-02-17T06:41:08.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj381 testdata.507902 2026-02-17T06:41:08.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj382 testdata.507902 2026-02-17T06:41:08.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.669 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj383 testdata.507902 2026-02-17T06:41:08.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj384 testdata.507902 2026-02-17T06:41:08.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj385 testdata.507902 2026-02-17T06:41:08.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.741 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj386 testdata.507902 2026-02-17T06:41:08.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj387 testdata.507902 2026-02-17T06:41:08.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj388 testdata.507902 2026-02-17T06:41:08.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj389 testdata.507902 2026-02-17T06:41:08.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj390 testdata.507902 2026-02-17T06:41:08.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj391 testdata.507902 2026-02-17T06:41:08.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj392 testdata.507902 2026-02-17T06:41:08.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj393 testdata.507902 2026-02-17T06:41:08.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj394 testdata.507902 2026-02-17T06:41:08.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj395 testdata.507902 2026-02-17T06:41:08.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj396 testdata.507902 2026-02-17T06:41:08.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:08.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj397 testdata.507902 2026-02-17T06:41:09.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj398 testdata.507902 2026-02-17T06:41:09.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj399 testdata.507902 2026-02-17T06:41:09.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj400 testdata.507902 2026-02-17T06:41:09.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj401 testdata.507902 2026-02-17T06:41:09.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj402 testdata.507902 2026-02-17T06:41:09.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.134 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj403 testdata.507902 2026-02-17T06:41:09.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj404 testdata.507902 2026-02-17T06:41:09.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj405 testdata.507902 2026-02-17T06:41:09.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj406 testdata.507902 2026-02-17T06:41:09.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj407 testdata.507902 2026-02-17T06:41:09.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj408 testdata.507902 2026-02-17T06:41:09.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj409 testdata.507902 2026-02-17T06:41:09.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj410 testdata.507902 2026-02-17T06:41:09.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj411 testdata.507902 2026-02-17T06:41:09.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj412 testdata.507902 2026-02-17T06:41:09.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj413 testdata.507902 2026-02-17T06:41:09.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj414 testdata.507902 2026-02-17T06:41:09.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj415 testdata.507902 2026-02-17T06:41:09.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj416 testdata.507902 2026-02-17T06:41:09.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj417 testdata.507902 2026-02-17T06:41:09.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj418 testdata.507902 2026-02-17T06:41:09.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj419 testdata.507902 2026-02-17T06:41:09.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj420 testdata.507902 2026-02-17T06:41:09.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj421 testdata.507902 2026-02-17T06:41:09.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj422 testdata.507902 2026-02-17T06:41:09.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj423 testdata.507902 2026-02-17T06:41:09.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj424 testdata.507902 2026-02-17T06:41:09.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.642 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj425 testdata.507902 2026-02-17T06:41:09.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj426 testdata.507902 2026-02-17T06:41:09.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj427 testdata.507902 2026-02-17T06:41:09.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj428 testdata.507902 2026-02-17T06:41:09.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj429 testdata.507902 2026-02-17T06:41:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj430 testdata.507902 2026-02-17T06:41:09.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj431 testdata.507902 2026-02-17T06:41:09.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj432 testdata.507902 2026-02-17T06:41:09.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.827 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj433 testdata.507902 2026-02-17T06:41:09.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj434 testdata.507902 2026-02-17T06:41:09.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj435 testdata.507902 2026-02-17T06:41:09.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj436 testdata.507902 2026-02-17T06:41:09.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj437 testdata.507902 2026-02-17T06:41:09.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj438 testdata.507902 2026-02-17T06:41:09.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj439 testdata.507902 2026-02-17T06:41:09.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:09.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj440 testdata.507902 2026-02-17T06:41:10.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj441 testdata.507902 2026-02-17T06:41:10.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj442 testdata.507902 2026-02-17T06:41:10.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj443 testdata.507902 2026-02-17T06:41:10.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj444 testdata.507902 2026-02-17T06:41:10.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.117 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj445 testdata.507902 2026-02-17T06:41:10.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj446 testdata.507902 2026-02-17T06:41:10.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj447 testdata.507902 2026-02-17T06:41:10.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj448 testdata.507902 2026-02-17T06:41:10.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj449 testdata.507902 2026-02-17T06:41:10.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj450 testdata.507902 2026-02-17T06:41:10.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj451 testdata.507902 2026-02-17T06:41:10.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj452 testdata.507902 2026-02-17T06:41:10.300 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj453 testdata.507902 2026-02-17T06:41:10.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj454 testdata.507902 2026-02-17T06:41:10.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj455 testdata.507902 2026-02-17T06:41:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj456 testdata.507902 2026-02-17T06:41:10.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj457 testdata.507902 2026-02-17T06:41:10.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj458 testdata.507902 2026-02-17T06:41:10.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj459 testdata.507902 2026-02-17T06:41:10.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj460 testdata.507902 2026-02-17T06:41:10.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj461 testdata.507902 2026-02-17T06:41:10.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj462 testdata.507902 2026-02-17T06:41:10.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj463 testdata.507902 2026-02-17T06:41:10.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj464 testdata.507902 2026-02-17T06:41:10.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj465 testdata.507902 2026-02-17T06:41:10.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj466 testdata.507902 2026-02-17T06:41:10.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj467 testdata.507902 2026-02-17T06:41:10.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj468 testdata.507902 2026-02-17T06:41:10.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj469 testdata.507902 2026-02-17T06:41:10.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj470 testdata.507902 2026-02-17T06:41:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj471 testdata.507902 2026-02-17T06:41:10.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj472 testdata.507902 2026-02-17T06:41:10.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj473 testdata.507902 2026-02-17T06:41:10.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj474 testdata.507902 2026-02-17T06:41:10.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj475 testdata.507902 2026-02-17T06:41:10.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj476 testdata.507902 2026-02-17T06:41:10.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj477 testdata.507902 2026-02-17T06:41:10.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj478 testdata.507902 2026-02-17T06:41:10.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj479 testdata.507902 2026-02-17T06:41:10.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj480 testdata.507902 2026-02-17T06:41:10.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj481 testdata.507902 2026-02-17T06:41:10.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj482 testdata.507902 2026-02-17T06:41:10.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:10.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj483 testdata.507902 2026-02-17T06:41:11.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj484 testdata.507902 2026-02-17T06:41:11.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj485 testdata.507902 2026-02-17T06:41:11.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj486 testdata.507902 2026-02-17T06:41:11.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj487 testdata.507902 2026-02-17T06:41:11.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj488 testdata.507902 2026-02-17T06:41:11.123 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj489 testdata.507902 2026-02-17T06:41:11.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj490 testdata.507902 2026-02-17T06:41:11.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj491 testdata.507902 2026-02-17T06:41:11.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj492 testdata.507902 2026-02-17T06:41:11.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj493 testdata.507902 2026-02-17T06:41:11.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj494 testdata.507902 2026-02-17T06:41:11.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.265 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj495 testdata.507902 2026-02-17T06:41:11.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj496 testdata.507902 2026-02-17T06:41:11.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj497 testdata.507902 2026-02-17T06:41:11.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj498 testdata.507902 2026-02-17T06:41:11.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj499 testdata.507902 2026-02-17T06:41:11.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj500 testdata.507902 2026-02-17T06:41:11.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj501 testdata.507902 2026-02-17T06:41:11.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj502 testdata.507902 2026-02-17T06:41:11.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj503 testdata.507902 2026-02-17T06:41:11.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj504 testdata.507902 2026-02-17T06:41:11.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.501 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj505 testdata.507902 2026-02-17T06:41:11.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj506 testdata.507902 2026-02-17T06:41:11.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj507 testdata.507902 2026-02-17T06:41:11.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj508 testdata.507902 2026-02-17T06:41:11.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj509 testdata.507902 2026-02-17T06:41:11.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj510 testdata.507902 2026-02-17T06:41:11.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj511 testdata.507902 2026-02-17T06:41:11.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj512 testdata.507902 2026-02-17T06:41:11.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj513 testdata.507902 2026-02-17T06:41:11.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj514 testdata.507902 2026-02-17T06:41:11.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj515 testdata.507902 2026-02-17T06:41:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj516 testdata.507902 2026-02-17T06:41:11.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj517 testdata.507902 2026-02-17T06:41:11.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj518 testdata.507902 2026-02-17T06:41:11.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj519 testdata.507902 2026-02-17T06:41:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj520 testdata.507902 2026-02-17T06:41:11.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj521 testdata.507902 2026-02-17T06:41:11.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj522 testdata.507902 2026-02-17T06:41:11.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj523 testdata.507902 2026-02-17T06:41:11.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj524 testdata.507902 2026-02-17T06:41:11.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:11.978 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj525 testdata.507902 2026-02-17T06:41:12.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj526 testdata.507902 2026-02-17T06:41:12.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj527 testdata.507902 2026-02-17T06:41:12.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj528 testdata.507902 2026-02-17T06:41:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj529 testdata.507902 2026-02-17T06:41:12.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj530 testdata.507902 2026-02-17T06:41:12.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj531 testdata.507902 2026-02-17T06:41:12.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj532 testdata.507902 2026-02-17T06:41:12.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj533 testdata.507902 2026-02-17T06:41:12.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj534 testdata.507902 2026-02-17T06:41:12.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj535 testdata.507902 2026-02-17T06:41:12.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj536 testdata.507902 2026-02-17T06:41:12.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj537 testdata.507902 2026-02-17T06:41:12.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj538 testdata.507902 2026-02-17T06:41:12.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj539 testdata.507902 2026-02-17T06:41:12.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj540 testdata.507902 2026-02-17T06:41:12.360 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj541 testdata.507902 2026-02-17T06:41:12.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj542 testdata.507902 2026-02-17T06:41:12.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj543 testdata.507902 2026-02-17T06:41:12.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj544 testdata.507902 2026-02-17T06:41:12.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj545 testdata.507902 2026-02-17T06:41:12.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj546 testdata.507902 2026-02-17T06:41:12.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj547 testdata.507902 2026-02-17T06:41:12.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj548 testdata.507902 2026-02-17T06:41:12.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj549 testdata.507902 2026-02-17T06:41:12.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj550 testdata.507902 2026-02-17T06:41:12.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj551 testdata.507902 2026-02-17T06:41:12.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj552 testdata.507902 2026-02-17T06:41:12.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj553 testdata.507902 2026-02-17T06:41:12.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj554 testdata.507902 2026-02-17T06:41:12.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj555 testdata.507902 2026-02-17T06:41:12.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj556 testdata.507902 2026-02-17T06:41:12.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj557 testdata.507902 2026-02-17T06:41:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj558 testdata.507902 2026-02-17T06:41:12.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj559 testdata.507902 2026-02-17T06:41:12.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj560 testdata.507902 2026-02-17T06:41:12.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj561 testdata.507902 2026-02-17T06:41:12.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj562 testdata.507902 2026-02-17T06:41:12.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj563 testdata.507902 2026-02-17T06:41:12.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj564 testdata.507902 2026-02-17T06:41:12.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj565 testdata.507902 2026-02-17T06:41:12.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj566 testdata.507902 2026-02-17T06:41:12.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:12.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj567 testdata.507902 2026-02-17T06:41:13.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj568 testdata.507902 2026-02-17T06:41:13.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj569 testdata.507902 2026-02-17T06:41:13.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj570 testdata.507902 2026-02-17T06:41:13.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj571 testdata.507902 2026-02-17T06:41:13.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj572 testdata.507902 2026-02-17T06:41:13.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj573 testdata.507902 2026-02-17T06:41:13.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj574 testdata.507902 2026-02-17T06:41:13.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj575 testdata.507902 2026-02-17T06:41:13.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj576 testdata.507902 2026-02-17T06:41:13.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj577 testdata.507902 2026-02-17T06:41:13.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj578 testdata.507902 2026-02-17T06:41:13.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj579 testdata.507902 2026-02-17T06:41:13.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj580 testdata.507902 2026-02-17T06:41:13.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj581 testdata.507902 2026-02-17T06:41:13.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj582 testdata.507902 2026-02-17T06:41:13.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj583 testdata.507902 2026-02-17T06:41:13.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj584 testdata.507902 2026-02-17T06:41:13.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj585 testdata.507902 2026-02-17T06:41:13.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj586 testdata.507902 2026-02-17T06:41:13.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj587 testdata.507902 2026-02-17T06:41:13.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj588 testdata.507902 2026-02-17T06:41:13.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj589 testdata.507902 2026-02-17T06:41:13.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj590 testdata.507902 2026-02-17T06:41:13.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj591 testdata.507902 2026-02-17T06:41:13.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj592 testdata.507902 2026-02-17T06:41:13.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj593 testdata.507902 2026-02-17T06:41:13.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj594 testdata.507902 2026-02-17T06:41:13.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj595 testdata.507902 2026-02-17T06:41:13.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj596 testdata.507902 2026-02-17T06:41:13.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj597 testdata.507902 2026-02-17T06:41:13.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj598 testdata.507902 2026-02-17T06:41:13.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj599 testdata.507902 2026-02-17T06:41:13.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj600 testdata.507902 2026-02-17T06:41:13.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj601 testdata.507902 2026-02-17T06:41:13.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj602 testdata.507902 2026-02-17T06:41:13.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj603 testdata.507902 2026-02-17T06:41:13.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj604 testdata.507902 2026-02-17T06:41:13.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj605 testdata.507902 2026-02-17T06:41:13.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj606 testdata.507902 2026-02-17T06:41:13.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj607 testdata.507902 2026-02-17T06:41:13.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj608 testdata.507902 2026-02-17T06:41:13.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj609 testdata.507902 2026-02-17T06:41:13.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj610 testdata.507902 2026-02-17T06:41:13.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:13.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj611 testdata.507902 2026-02-17T06:41:14.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj612 testdata.507902 2026-02-17T06:41:14.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj613 testdata.507902 2026-02-17T06:41:14.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj614 testdata.507902 2026-02-17T06:41:14.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj615 testdata.507902 2026-02-17T06:41:14.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj616 testdata.507902 2026-02-17T06:41:14.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj617 testdata.507902 2026-02-17T06:41:14.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj618 testdata.507902 2026-02-17T06:41:14.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj619 testdata.507902 2026-02-17T06:41:14.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj620 testdata.507902 2026-02-17T06:41:14.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.232 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj621 testdata.507902 2026-02-17T06:41:14.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj622 testdata.507902 2026-02-17T06:41:14.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj623 testdata.507902 2026-02-17T06:41:14.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj624 testdata.507902 2026-02-17T06:41:14.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj625 testdata.507902 2026-02-17T06:41:14.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj626 testdata.507902 2026-02-17T06:41:14.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj627 testdata.507902 2026-02-17T06:41:14.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj628 testdata.507902 2026-02-17T06:41:14.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.426 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj629 testdata.507902 2026-02-17T06:41:14.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj630 testdata.507902 2026-02-17T06:41:14.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj631 testdata.507902 2026-02-17T06:41:14.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj632 testdata.507902 2026-02-17T06:41:14.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj633 testdata.507902 2026-02-17T06:41:14.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj634 testdata.507902 2026-02-17T06:41:14.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj635 testdata.507902 2026-02-17T06:41:14.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.581 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj636 testdata.507902 2026-02-17T06:41:14.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj637 testdata.507902 2026-02-17T06:41:14.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj638 testdata.507902 2026-02-17T06:41:14.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj639 testdata.507902 2026-02-17T06:41:14.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj640 testdata.507902 2026-02-17T06:41:14.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj641 testdata.507902 2026-02-17T06:41:14.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj642 testdata.507902 2026-02-17T06:41:14.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj643 testdata.507902 2026-02-17T06:41:14.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj644 testdata.507902 2026-02-17T06:41:14.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj645 testdata.507902 2026-02-17T06:41:14.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj646 testdata.507902 2026-02-17T06:41:14.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj647 testdata.507902 2026-02-17T06:41:14.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj648 testdata.507902 2026-02-17T06:41:14.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj649 testdata.507902 2026-02-17T06:41:14.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj650 testdata.507902 2026-02-17T06:41:14.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.946 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj651 testdata.507902 2026-02-17T06:41:14.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj652 testdata.507902 2026-02-17T06:41:14.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:14.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj653 testdata.507902 2026-02-17T06:41:15.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj654 testdata.507902 2026-02-17T06:41:15.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj655 testdata.507902 2026-02-17T06:41:15.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj656 testdata.507902 2026-02-17T06:41:15.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj657 testdata.507902 2026-02-17T06:41:15.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj658 testdata.507902 2026-02-17T06:41:15.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj659 testdata.507902 2026-02-17T06:41:15.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj660 testdata.507902 2026-02-17T06:41:15.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj661 testdata.507902 2026-02-17T06:41:15.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj662 testdata.507902 2026-02-17T06:41:15.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj663 testdata.507902 2026-02-17T06:41:15.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj664 testdata.507902 2026-02-17T06:41:15.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj665 testdata.507902 2026-02-17T06:41:15.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj666 testdata.507902 2026-02-17T06:41:15.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj667 testdata.507902 2026-02-17T06:41:15.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj668 testdata.507902 2026-02-17T06:41:15.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj669 testdata.507902 2026-02-17T06:41:15.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj670 testdata.507902 2026-02-17T06:41:15.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj671 testdata.507902 2026-02-17T06:41:15.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.435 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj672 testdata.507902 2026-02-17T06:41:15.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj673 testdata.507902 2026-02-17T06:41:15.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj674 testdata.507902 2026-02-17T06:41:15.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.508 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj675 testdata.507902 2026-02-17T06:41:15.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj676 testdata.507902 2026-02-17T06:41:15.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj677 testdata.507902 2026-02-17T06:41:15.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj678 testdata.507902 2026-02-17T06:41:15.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj679 testdata.507902 2026-02-17T06:41:15.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj680 testdata.507902 2026-02-17T06:41:15.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.653 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj681 testdata.507902 2026-02-17T06:41:15.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj682 testdata.507902 2026-02-17T06:41:15.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj683 testdata.507902 2026-02-17T06:41:15.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj684 testdata.507902 2026-02-17T06:41:15.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj685 testdata.507902 2026-02-17T06:41:15.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj686 testdata.507902 2026-02-17T06:41:15.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj687 testdata.507902 2026-02-17T06:41:15.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj688 testdata.507902 2026-02-17T06:41:15.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj689 testdata.507902 2026-02-17T06:41:15.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj690 testdata.507902 2026-02-17T06:41:15.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj691 testdata.507902 2026-02-17T06:41:15.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj692 testdata.507902 2026-02-17T06:41:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj693 testdata.507902 2026-02-17T06:41:15.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj694 testdata.507902 2026-02-17T06:41:15.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:15.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj695 testdata.507902 2026-02-17T06:41:16.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj696 testdata.507902 2026-02-17T06:41:16.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj697 testdata.507902 2026-02-17T06:41:16.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj698 testdata.507902 2026-02-17T06:41:16.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj699 testdata.507902 2026-02-17T06:41:16.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj700 testdata.507902 2026-02-17T06:41:16.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj701 testdata.507902 2026-02-17T06:41:16.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj702 testdata.507902 2026-02-17T06:41:16.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj703 testdata.507902 2026-02-17T06:41:16.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj704 testdata.507902 2026-02-17T06:41:16.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj705 testdata.507902 2026-02-17T06:41:16.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj706 testdata.507902 2026-02-17T06:41:16.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj707 testdata.507902 2026-02-17T06:41:16.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj708 testdata.507902 2026-02-17T06:41:16.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj709 testdata.507902 2026-02-17T06:41:16.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj710 testdata.507902 2026-02-17T06:41:16.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj711 testdata.507902 2026-02-17T06:41:16.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj712 testdata.507902 2026-02-17T06:41:16.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj713 testdata.507902 2026-02-17T06:41:16.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj714 testdata.507902 2026-02-17T06:41:16.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj715 testdata.507902 2026-02-17T06:41:16.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj716 testdata.507902 2026-02-17T06:41:16.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj717 testdata.507902 2026-02-17T06:41:16.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj718 testdata.507902 2026-02-17T06:41:16.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj719 testdata.507902 2026-02-17T06:41:16.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj720 testdata.507902 2026-02-17T06:41:16.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj721 testdata.507902 2026-02-17T06:41:16.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj722 testdata.507902 2026-02-17T06:41:16.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj723 testdata.507902 2026-02-17T06:41:16.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj724 testdata.507902 2026-02-17T06:41:16.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj725 testdata.507902 2026-02-17T06:41:16.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj726 testdata.507902 2026-02-17T06:41:16.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj727 testdata.507902 2026-02-17T06:41:16.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj728 testdata.507902 2026-02-17T06:41:16.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj729 testdata.507902 2026-02-17T06:41:16.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj730 testdata.507902 2026-02-17T06:41:16.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj731 testdata.507902 2026-02-17T06:41:16.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj732 testdata.507902 2026-02-17T06:41:16.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.855 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj733 testdata.507902 2026-02-17T06:41:16.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj734 testdata.507902 2026-02-17T06:41:16.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj735 testdata.507902 2026-02-17T06:41:16.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj736 testdata.507902 2026-02-17T06:41:16.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.953 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj737 testdata.507902 2026-02-17T06:41:16.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:16.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj738 testdata.507902 2026-02-17T06:41:17.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj739 testdata.507902 2026-02-17T06:41:17.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj740 testdata.507902 2026-02-17T06:41:17.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj741 testdata.507902 2026-02-17T06:41:17.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj742 testdata.507902 2026-02-17T06:41:17.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj743 testdata.507902 2026-02-17T06:41:17.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj744 testdata.507902 2026-02-17T06:41:17.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj745 testdata.507902 2026-02-17T06:41:17.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj746 testdata.507902 2026-02-17T06:41:17.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj747 testdata.507902 2026-02-17T06:41:17.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj748 testdata.507902 2026-02-17T06:41:17.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj749 testdata.507902 2026-02-17T06:41:17.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj750 testdata.507902 2026-02-17T06:41:17.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj751 testdata.507902 2026-02-17T06:41:17.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj752 testdata.507902 2026-02-17T06:41:17.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj753 testdata.507902 2026-02-17T06:41:17.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj754 testdata.507902 2026-02-17T06:41:17.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj755 testdata.507902 2026-02-17T06:41:17.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj756 testdata.507902 2026-02-17T06:41:17.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj757 testdata.507902 2026-02-17T06:41:17.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.429 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj758 testdata.507902 2026-02-17T06:41:17.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj759 testdata.507902 2026-02-17T06:41:17.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj760 testdata.507902 2026-02-17T06:41:17.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.502 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj761 testdata.507902 2026-02-17T06:41:17.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj762 testdata.507902 2026-02-17T06:41:17.550 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj763 testdata.507902 2026-02-17T06:41:17.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj764 testdata.507902 2026-02-17T06:41:17.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj765 testdata.507902 2026-02-17T06:41:17.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj766 testdata.507902 2026-02-17T06:41:17.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj767 testdata.507902 2026-02-17T06:41:17.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj768 testdata.507902 2026-02-17T06:41:17.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj769 testdata.507902 2026-02-17T06:41:17.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj770 testdata.507902 2026-02-17T06:41:17.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj771 testdata.507902 2026-02-17T06:41:17.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj772 testdata.507902 2026-02-17T06:41:17.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj773 testdata.507902 2026-02-17T06:41:17.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj774 testdata.507902 2026-02-17T06:41:17.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj775 testdata.507902 2026-02-17T06:41:17.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj776 testdata.507902 2026-02-17T06:41:17.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj777 testdata.507902 2026-02-17T06:41:17.899 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj778 testdata.507902 2026-02-17T06:41:17.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj779 testdata.507902 2026-02-17T06:41:17.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj780 testdata.507902 2026-02-17T06:41:17.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj781 testdata.507902 2026-02-17T06:41:17.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:17.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj782 testdata.507902 2026-02-17T06:41:18.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj783 testdata.507902 2026-02-17T06:41:18.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj784 testdata.507902 2026-02-17T06:41:18.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj785 testdata.507902 2026-02-17T06:41:18.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj786 testdata.507902 2026-02-17T06:41:18.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj787 testdata.507902 2026-02-17T06:41:18.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj788 testdata.507902 2026-02-17T06:41:18.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj789 testdata.507902 2026-02-17T06:41:18.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.176 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj790 testdata.507902 2026-02-17T06:41:18.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj791 testdata.507902 2026-02-17T06:41:18.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj792 testdata.507902 2026-02-17T06:41:18.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj793 testdata.507902 2026-02-17T06:41:18.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj794 testdata.507902 2026-02-17T06:41:18.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj795 testdata.507902 2026-02-17T06:41:18.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj796 testdata.507902 2026-02-17T06:41:18.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj797 testdata.507902 2026-02-17T06:41:18.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj798 testdata.507902 2026-02-17T06:41:18.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj799 testdata.507902 2026-02-17T06:41:18.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj800 testdata.507902 2026-02-17T06:41:18.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.439 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj801 testdata.507902 2026-02-17T06:41:18.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj802 testdata.507902 2026-02-17T06:41:18.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj803 testdata.507902 2026-02-17T06:41:18.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj804 testdata.507902 2026-02-17T06:41:18.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj805 testdata.507902 2026-02-17T06:41:18.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.561 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj806 testdata.507902 2026-02-17T06:41:18.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj807 testdata.507902 2026-02-17T06:41:18.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj808 testdata.507902 2026-02-17T06:41:18.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj809 testdata.507902 2026-02-17T06:41:18.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.648 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj810 testdata.507902 2026-02-17T06:41:18.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj811 testdata.507902 2026-02-17T06:41:18.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj812 testdata.507902 2026-02-17T06:41:18.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj813 testdata.507902 2026-02-17T06:41:18.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj814 testdata.507902 2026-02-17T06:41:18.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj815 testdata.507902 2026-02-17T06:41:18.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj816 testdata.507902 2026-02-17T06:41:18.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj817 testdata.507902 2026-02-17T06:41:18.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj818 testdata.507902 2026-02-17T06:41:18.838 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj819 testdata.507902 2026-02-17T06:41:18.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj820 testdata.507902 2026-02-17T06:41:18.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj821 testdata.507902 2026-02-17T06:41:18.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj822 testdata.507902 2026-02-17T06:41:18.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj823 testdata.507902 2026-02-17T06:41:18.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj824 testdata.507902 2026-02-17T06:41:18.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:18.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj825 testdata.507902 2026-02-17T06:41:19.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj826 testdata.507902 2026-02-17T06:41:19.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj827 testdata.507902 2026-02-17T06:41:19.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj828 testdata.507902 2026-02-17T06:41:19.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj829 testdata.507902 2026-02-17T06:41:19.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj830 testdata.507902 2026-02-17T06:41:19.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.121 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj831 testdata.507902 2026-02-17T06:41:19.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj832 testdata.507902 2026-02-17T06:41:19.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj833 testdata.507902 2026-02-17T06:41:19.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj834 testdata.507902 2026-02-17T06:41:19.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj835 testdata.507902 2026-02-17T06:41:19.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj836 testdata.507902 2026-02-17T06:41:19.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj837 testdata.507902 2026-02-17T06:41:19.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj838 testdata.507902 2026-02-17T06:41:19.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj839 testdata.507902 2026-02-17T06:41:19.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj840 testdata.507902 2026-02-17T06:41:19.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj841 testdata.507902 2026-02-17T06:41:19.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj842 testdata.507902 2026-02-17T06:41:19.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj843 testdata.507902 2026-02-17T06:41:19.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj844 testdata.507902 2026-02-17T06:41:19.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj845 testdata.507902 2026-02-17T06:41:19.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj846 testdata.507902 2026-02-17T06:41:19.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj847 testdata.507902 2026-02-17T06:41:19.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj848 testdata.507902 2026-02-17T06:41:19.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.560 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj849 testdata.507902 2026-02-17T06:41:19.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj850 testdata.507902 2026-02-17T06:41:19.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj851 testdata.507902 2026-02-17T06:41:19.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj852 testdata.507902 2026-02-17T06:41:19.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj853 testdata.507902 2026-02-17T06:41:19.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj854 testdata.507902 2026-02-17T06:41:19.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj855 testdata.507902 2026-02-17T06:41:19.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj856 testdata.507902 2026-02-17T06:41:19.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj857 testdata.507902 2026-02-17T06:41:19.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.782 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj858 testdata.507902 2026-02-17T06:41:19.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj859 testdata.507902 2026-02-17T06:41:19.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj860 testdata.507902 2026-02-17T06:41:19.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj861 testdata.507902 2026-02-17T06:41:19.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.872 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj862 testdata.507902 2026-02-17T06:41:19.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj863 testdata.507902 2026-02-17T06:41:19.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj864 testdata.507902 2026-02-17T06:41:19.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj865 testdata.507902 2026-02-17T06:41:19.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj866 testdata.507902 2026-02-17T06:41:19.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:19.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj867 testdata.507902 2026-02-17T06:41:20.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj868 testdata.507902 2026-02-17T06:41:20.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj869 testdata.507902 2026-02-17T06:41:20.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj870 testdata.507902 2026-02-17T06:41:20.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj871 testdata.507902 2026-02-17T06:41:20.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj872 testdata.507902 2026-02-17T06:41:20.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.133 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj873 testdata.507902 2026-02-17T06:41:20.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj874 testdata.507902 2026-02-17T06:41:20.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj875 testdata.507902 2026-02-17T06:41:20.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj876 testdata.507902 2026-02-17T06:41:20.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj877 testdata.507902 2026-02-17T06:41:20.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj878 testdata.507902 2026-02-17T06:41:20.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.264 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj879 testdata.507902 2026-02-17T06:41:20.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.287 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj880 testdata.507902 2026-02-17T06:41:20.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj881 testdata.507902 2026-02-17T06:41:20.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj882 testdata.507902 2026-02-17T06:41:20.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.361 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj883 testdata.507902 2026-02-17T06:41:20.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj884 testdata.507902 2026-02-17T06:41:20.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj885 testdata.507902 2026-02-17T06:41:20.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj886 testdata.507902 2026-02-17T06:41:20.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj887 testdata.507902 2026-02-17T06:41:20.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj888 testdata.507902 2026-02-17T06:41:20.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj889 testdata.507902 2026-02-17T06:41:20.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj890 testdata.507902 2026-02-17T06:41:20.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj891 testdata.507902 2026-02-17T06:41:20.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj892 testdata.507902 2026-02-17T06:41:20.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj893 testdata.507902 2026-02-17T06:41:20.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj894 testdata.507902 2026-02-17T06:41:20.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj895 testdata.507902 2026-02-17T06:41:20.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj896 testdata.507902 2026-02-17T06:41:20.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj897 testdata.507902 2026-02-17T06:41:20.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj898 testdata.507902 2026-02-17T06:41:20.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj899 testdata.507902 2026-02-17T06:41:20.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj900 testdata.507902 2026-02-17T06:41:20.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj901 testdata.507902 2026-02-17T06:41:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj902 testdata.507902 2026-02-17T06:41:20.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj903 testdata.507902 2026-02-17T06:41:20.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj904 testdata.507902 2026-02-17T06:41:20.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj905 testdata.507902 2026-02-17T06:41:20.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj906 testdata.507902 2026-02-17T06:41:20.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj907 testdata.507902 2026-02-17T06:41:20.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.947 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj908 testdata.507902 2026-02-17T06:41:20.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj909 testdata.507902 2026-02-17T06:41:20.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:20.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj910 testdata.507902 2026-02-17T06:41:21.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj911 testdata.507902 2026-02-17T06:41:21.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj912 testdata.507902 2026-02-17T06:41:21.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj913 testdata.507902 2026-02-17T06:41:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj914 testdata.507902 2026-02-17T06:41:21.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj915 testdata.507902 2026-02-17T06:41:21.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj916 testdata.507902 2026-02-17T06:41:21.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj917 testdata.507902 2026-02-17T06:41:21.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj918 testdata.507902 2026-02-17T06:41:21.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj919 testdata.507902 2026-02-17T06:41:21.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj920 testdata.507902 2026-02-17T06:41:21.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.261 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj921 testdata.507902 2026-02-17T06:41:21.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj922 testdata.507902 2026-02-17T06:41:21.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj923 testdata.507902 2026-02-17T06:41:21.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj924 testdata.507902 2026-02-17T06:41:21.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj925 testdata.507902 2026-02-17T06:41:21.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj926 testdata.507902 2026-02-17T06:41:21.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj927 testdata.507902 2026-02-17T06:41:21.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj928 testdata.507902 2026-02-17T06:41:21.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj929 testdata.507902 2026-02-17T06:41:21.465 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj930 testdata.507902 2026-02-17T06:41:21.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj931 testdata.507902 2026-02-17T06:41:21.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.511 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj932 testdata.507902 2026-02-17T06:41:21.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj933 testdata.507902 2026-02-17T06:41:21.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj934 testdata.507902 2026-02-17T06:41:21.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj935 testdata.507902 2026-02-17T06:41:21.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj936 testdata.507902 2026-02-17T06:41:21.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj937 testdata.507902 2026-02-17T06:41:21.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj938 testdata.507902 2026-02-17T06:41:21.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj939 testdata.507902 2026-02-17T06:41:21.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj940 testdata.507902 2026-02-17T06:41:21.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj941 testdata.507902 2026-02-17T06:41:21.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj942 testdata.507902 2026-02-17T06:41:21.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj943 testdata.507902 2026-02-17T06:41:21.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj944 testdata.507902 2026-02-17T06:41:21.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj945 testdata.507902 2026-02-17T06:41:21.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.829 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj946 testdata.507902 2026-02-17T06:41:21.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj947 testdata.507902 2026-02-17T06:41:21.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj948 testdata.507902 2026-02-17T06:41:21.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj949 testdata.507902 2026-02-17T06:41:21.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj950 testdata.507902 2026-02-17T06:41:21.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj951 testdata.507902 2026-02-17T06:41:21.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj952 testdata.507902 2026-02-17T06:41:21.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:21.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj953 testdata.507902 2026-02-17T06:41:22.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj954 testdata.507902 2026-02-17T06:41:22.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj955 testdata.507902 2026-02-17T06:41:22.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj956 testdata.507902 2026-02-17T06:41:22.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj957 testdata.507902 2026-02-17T06:41:22.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj958 testdata.507902 2026-02-17T06:41:22.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj959 testdata.507902 2026-02-17T06:41:22.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj960 testdata.507902 2026-02-17T06:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj961 testdata.507902 2026-02-17T06:41:22.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj962 testdata.507902 2026-02-17T06:41:22.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj963 testdata.507902 2026-02-17T06:41:22.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj964 testdata.507902 2026-02-17T06:41:22.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj965 testdata.507902 2026-02-17T06:41:22.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj966 testdata.507902 2026-02-17T06:41:22.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj967 testdata.507902 2026-02-17T06:41:22.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.341 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj968 testdata.507902 2026-02-17T06:41:22.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj969 testdata.507902 2026-02-17T06:41:22.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj970 testdata.507902 2026-02-17T06:41:22.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj971 testdata.507902 2026-02-17T06:41:22.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj972 testdata.507902 2026-02-17T06:41:22.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj973 testdata.507902 2026-02-17T06:41:22.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj974 testdata.507902 2026-02-17T06:41:22.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj975 testdata.507902 2026-02-17T06:41:22.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj976 testdata.507902 2026-02-17T06:41:22.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj977 testdata.507902 2026-02-17T06:41:22.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj978 testdata.507902 2026-02-17T06:41:22.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj979 testdata.507902 2026-02-17T06:41:22.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj980 testdata.507902 2026-02-17T06:41:22.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj981 testdata.507902 2026-02-17T06:41:22.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj982 testdata.507902 2026-02-17T06:41:22.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj983 testdata.507902 2026-02-17T06:41:22.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj984 testdata.507902 2026-02-17T06:41:22.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj985 testdata.507902 2026-02-17T06:41:22.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj986 testdata.507902 2026-02-17T06:41:22.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj987 testdata.507902 2026-02-17T06:41:22.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj988 testdata.507902 2026-02-17T06:41:22.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj989 testdata.507902 2026-02-17T06:41:22.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj990 testdata.507902 2026-02-17T06:41:22.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj991 testdata.507902 2026-02-17T06:41:22.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj992 testdata.507902 2026-02-17T06:41:22.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj993 testdata.507902 2026-02-17T06:41:22.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj994 testdata.507902 2026-02-17T06:41:22.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj995 testdata.507902 2026-02-17T06:41:22.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:22.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj996 testdata.507902 2026-02-17T06:41:23.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:23.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj997 testdata.507902 2026-02-17T06:41:23.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:23.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj998 testdata.507902 2026-02-17T06:41:23.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:23.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj999 testdata.507902 2026-02-17T06:41:23.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T06:41:23.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1000 testdata.507902 2026-02-17T06:41:23.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:339: _scrub_abort: rm -f testdata.507902 2026-02-17T06:41:23.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: get_primary test obj1 2026-02-17T06:41:23.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:41:23.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:41:23.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:41:23.122 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:41:23.273 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: local primary=1 2026-02-17T06:41:23.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:342: _scrub_abort: local pgid=1.0 2026-02-17T06:41:23.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: ceph tell 1.0 schedule-scrub 2026-02-17T06:41:23.336 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:41:23.336 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:41:23.336 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:41:23.337 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:39:43.331339+0000" 2026-02-17T06:41:23.337 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:41:23.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: set -o pipefail 2026-02-17T06:41:23.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:348: _scrub_abort: found=no 2026-02-17T06:41:23.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: seq 0 200 2026-02-17T06:41:23.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:23.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T06:41:23.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:23.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:23.492 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:23.493 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:23.493 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:23.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:23.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:23.493 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:23.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836526 2026-02-17T06:41:23.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836526 2026-02-17T06:41:23.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526' 2026-02-17T06:41:23.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:23.563 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:23.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705701 2026-02-17T06:41:23.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705701 2026-02-17T06:41:23.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-38654705701' 2026-02-17T06:41:23.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:23.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:23.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542173 2026-02-17T06:41:23.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542173 2026-02-17T06:41:23.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-38654705701 2-60129542173' 2026-02-17T06:41:23.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:23.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836526 2026-02-17T06:41:23.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:23.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:23.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836526 2026-02-17T06:41:23.706 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:23.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836526 2026-02-17T06:41:23.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836526' 2026-02-17T06:41:23.708 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836526 2026-02-17T06:41:23.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:23.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836526 2026-02-17T06:41:23.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:23.866 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705701 2026-02-17T06:41:23.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:23.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:23.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705701 2026-02-17T06:41:23.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:23.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705701 2026-02-17T06:41:23.870 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705701 2026-02-17T06:41:23.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-17T06:41:23.871 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:24.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705701 -lt 38654705701 2026-02-17T06:41:24.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:24.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542173 2026-02-17T06:41:24.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:24.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:24.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542173 2026-02-17T06:41:24.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:24.023 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542173 2026-02-17T06:41:24.024 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542173 2026-02-17T06:41:24.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542173' 2026-02-17T06:41:24.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:24.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542174 -lt 60129542173 2026-02-17T06:41:24.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:24.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T06:41:24.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:24.320 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:24.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:24.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T06:41:24.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:24.330 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:24.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:24.479 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:24.479 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:24.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:24.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:24.479 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:24.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836528 2026-02-17T06:41:24.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836528 2026-02-17T06:41:24.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528' 2026-02-17T06:41:24.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:24.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:24.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705703 2026-02-17T06:41:24.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705703 2026-02-17T06:41:24.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705703' 2026-02-17T06:41:24.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:24.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:24.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542175 2026-02-17T06:41:24.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542175 2026-02-17T06:41:24.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705703 2-60129542175' 2026-02-17T06:41:24.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:24.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836528 2026-02-17T06:41:24.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:24.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:24.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836528 2026-02-17T06:41:24.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:24.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836528 2026-02-17T06:41:24.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836528' 2026-02-17T06:41:24.694 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836528 2026-02-17T06:41:24.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:24.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836528 2026-02-17T06:41:24.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:25.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:25.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:25.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836529 -lt 21474836528 2026-02-17T06:41:25.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:25.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705703 2026-02-17T06:41:25.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:25.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:25.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705703 2026-02-17T06:41:25.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:25.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705703 2026-02-17T06:41:25.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705703' 2026-02-17T06:41:25.999 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705703 2026-02-17T06:41:26.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:26.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705704 -lt 38654705703 2026-02-17T06:41:26.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:26.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542175 2026-02-17T06:41:26.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:26.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:26.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542175 2026-02-17T06:41:26.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:26.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542175 2026-02-17T06:41:26.155 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542175 2026-02-17T06:41:26.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542175' 2026-02-17T06:41:26.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:26.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542177 -lt 60129542175 2026-02-17T06:41:26.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:26.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T06:41:26.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:26.455 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:26.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:354: _scrub_abort: found=yes 2026-02-17T06:41:26.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:356: _scrub_abort: break 2026-02-17T06:41:26.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:359: _scrub_abort: set +o pipefail 2026-02-17T06:41:26.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:361: _scrub_abort: test yes = no 2026-02-17T06:41:26.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:367: _scrub_abort: ceph osd set noscrub 2026-02-17T06:41:26.673 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is set 2026-02-17T06:41:26.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:368: _scrub_abort: '[' scrub = deep-scrub ']' 2026-02-17T06:41:26.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:374: _scrub_abort: set -o pipefail 2026-02-17T06:41:26.685 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: seq 0 200 2026-02-17T06:41:26.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:26.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:26.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:26.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:26.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:26.835 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:26.835 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:26.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:26.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:26.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:26.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836531 2026-02-17T06:41:26.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836531 2026-02-17T06:41:26.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531' 2026-02-17T06:41:26.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:26.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:26.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705706 2026-02-17T06:41:26.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705706 2026-02-17T06:41:26.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-38654705706' 2026-02-17T06:41:26.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:26.975 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:27.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542179 2026-02-17T06:41:27.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542179 2026-02-17T06:41:27.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-38654705706 2-60129542179' 2026-02-17T06:41:27.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:27.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836531 2026-02-17T06:41:27.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:27.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:27.048 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836531 2026-02-17T06:41:27.049 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:27.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836531 2026-02-17T06:41:27.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836531' 2026-02-17T06:41:27.051 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836531 2026-02-17T06:41:27.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:27.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836529 -lt 21474836531 2026-02-17T06:41:27.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:28.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:28.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:28.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836532 -lt 21474836531 2026-02-17T06:41:28.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:28.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705706 2026-02-17T06:41:28.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:28.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:28.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705706 2026-02-17T06:41:28.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:28.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705706 2026-02-17T06:41:28.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705706' 2026-02-17T06:41:28.352 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705706 2026-02-17T06:41:28.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:28.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705707 -lt 38654705706 2026-02-17T06:41:28.510 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:28.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542179 2026-02-17T06:41:28.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:28.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:28.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542179 2026-02-17T06:41:28.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:28.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542179 2026-02-17T06:41:28.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542179' 2026-02-17T06:41:28.514 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542179 2026-02-17T06:41:28.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:28.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542180 -lt 60129542179 2026-02-17T06:41:28.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:28.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:28.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:28.804 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:28.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:28.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:28.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:28.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:28.813 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:28.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:28.963 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:28.963 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:28.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:28.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:28.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:29.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836534 2026-02-17T06:41:29.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836534 2026-02-17T06:41:29.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534' 2026-02-17T06:41:29.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:29.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:29.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705710 2026-02-17T06:41:29.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705710 2026-02-17T06:41:29.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705710' 2026-02-17T06:41:29.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:29.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:29.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542182 2026-02-17T06:41:29.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542182 2026-02-17T06:41:29.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705710 2-60129542182' 2026-02-17T06:41:29.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:29.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836534 2026-02-17T06:41:29.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:29.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:29.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836534 2026-02-17T06:41:29.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:29.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836534 2026-02-17T06:41:29.182 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836534 2026-02-17T06:41:29.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836534' 2026-02-17T06:41:29.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:29.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836532 -lt 21474836534 2026-02-17T06:41:29.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:30.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:30.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:30.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836535 -lt 21474836534 2026-02-17T06:41:30.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:30.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705710 2026-02-17T06:41:30.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:30.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:30.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705710 2026-02-17T06:41:30.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:30.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705710 2026-02-17T06:41:30.487 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705710 2026-02-17T06:41:30.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705710' 2026-02-17T06:41:30.487 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:30.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705710 -lt 38654705710 2026-02-17T06:41:30.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:30.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542182 2026-02-17T06:41:30.648 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:30.649 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:30.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542182 2026-02-17T06:41:30.650 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:30.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542182 2026-02-17T06:41:30.651 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542182 2026-02-17T06:41:30.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542182' 2026-02-17T06:41:30.652 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:30.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542183 -lt 60129542182 2026-02-17T06:41:30.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:30.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:30.803 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:30.945 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:30.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:30.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:30.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:30.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:30.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:31.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:31.106 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:31.106 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:31.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:31.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:31.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:31.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836537 2026-02-17T06:41:31.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836537 2026-02-17T06:41:31.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537' 2026-02-17T06:41:31.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:31.175 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:31.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705713 2026-02-17T06:41:31.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705713 2026-02-17T06:41:31.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-38654705713' 2026-02-17T06:41:31.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:31.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:31.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542185 2026-02-17T06:41:31.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542185 2026-02-17T06:41:31.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-38654705713 2-60129542185' 2026-02-17T06:41:31.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:31.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836537 2026-02-17T06:41:31.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:31.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:31.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836537 2026-02-17T06:41:31.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:31.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836537 2026-02-17T06:41:31.320 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836537 2026-02-17T06:41:31.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836537' 2026-02-17T06:41:31.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836535 -lt 21474836537 2026-02-17T06:41:31.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:32.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:32.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:32.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836538 -lt 21474836537 2026-02-17T06:41:32.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:32.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705713 2026-02-17T06:41:32.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:32.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:32.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705713 2026-02-17T06:41:32.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:32.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705713 2026-02-17T06:41:32.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705713' 2026-02-17T06:41:32.634 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705713 2026-02-17T06:41:32.634 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:32.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705713 -lt 38654705713 2026-02-17T06:41:32.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:32.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542185 2026-02-17T06:41:32.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:32.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:32.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542185 2026-02-17T06:41:32.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:32.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542185 2026-02-17T06:41:32.787 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542185 2026-02-17T06:41:32.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542185' 2026-02-17T06:41:32.788 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:32.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542186 -lt 60129542185 2026-02-17T06:41:32.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:32.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:32.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:33.087 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:33.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:33.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:33.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:33.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:33.096 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:33.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:33.240 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:33.240 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:33.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:33.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:33.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:33.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836540 2026-02-17T06:41:33.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836540 2026-02-17T06:41:33.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540' 2026-02-17T06:41:33.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:33.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:33.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705716 2026-02-17T06:41:33.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705716 2026-02-17T06:41:33.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540 1-38654705716' 2026-02-17T06:41:33.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:33.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542188 2026-02-17T06:41:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542188 2026-02-17T06:41:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540 1-38654705716 2-60129542188' 2026-02-17T06:41:33.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:33.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836540 2026-02-17T06:41:33.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:33.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:33.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836540 2026-02-17T06:41:33.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:33.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836540 2026-02-17T06:41:33.459 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836540 2026-02-17T06:41:33.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836540' 2026-02-17T06:41:33.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:33.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836538 -lt 21474836540 2026-02-17T06:41:33.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:34.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:34.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:34.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836541 -lt 21474836540 2026-02-17T06:41:34.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:34.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705716 2026-02-17T06:41:34.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:34.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:34.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705716 2026-02-17T06:41:34.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:34.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705716 2026-02-17T06:41:34.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705716' 2026-02-17T06:41:34.776 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705716 2026-02-17T06:41:34.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:34.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705716 -lt 38654705716 2026-02-17T06:41:34.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:34.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542188 2026-02-17T06:41:34.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:34.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:34.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542188 2026-02-17T06:41:34.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:34.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542188 2026-02-17T06:41:34.933 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542188 2026-02-17T06:41:34.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542188' 2026-02-17T06:41:34.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:35.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542189 -lt 60129542188 2026-02-17T06:41:35.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:35.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:35.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:35.224 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:35.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:35.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:35.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:35.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:35.233 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:35.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:35.381 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:35.381 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:35.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:35.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:35.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:35.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836543 2026-02-17T06:41:35.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836543 2026-02-17T06:41:35.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543' 2026-02-17T06:41:35.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:35.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:35.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705719 2026-02-17T06:41:35.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705719 2026-02-17T06:41:35.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543 1-38654705719' 2026-02-17T06:41:35.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:35.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:35.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542191 2026-02-17T06:41:35.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542191 2026-02-17T06:41:35.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543 1-38654705719 2-60129542191' 2026-02-17T06:41:35.595 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:35.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836543 2026-02-17T06:41:35.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:35.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:35.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836543 2026-02-17T06:41:35.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:35.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836543 2026-02-17T06:41:35.598 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836543 2026-02-17T06:41:35.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836543' 2026-02-17T06:41:35.599 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:35.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836541 -lt 21474836543 2026-02-17T06:41:35.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:36.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:36.751 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:36.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836544 -lt 21474836543 2026-02-17T06:41:36.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:36.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705719 2026-02-17T06:41:36.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:36.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:36.900 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705719 2026-02-17T06:41:36.901 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:36.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705719 2026-02-17T06:41:36.902 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705719 2026-02-17T06:41:36.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705719' 2026-02-17T06:41:36.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:37.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705719 -lt 38654705719 2026-02-17T06:41:37.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:37.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542191 2026-02-17T06:41:37.053 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:37.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:37.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542191 2026-02-17T06:41:37.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:37.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542191 2026-02-17T06:41:37.056 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542191 2026-02-17T06:41:37.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542191' 2026-02-17T06:41:37.057 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:37.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542192 -lt 60129542191 2026-02-17T06:41:37.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:37.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:37.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:37.355 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:37.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:37.520 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:37.521 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:37.521 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:37.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:37.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:37.521 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:37.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836547 2026-02-17T06:41:37.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836547 2026-02-17T06:41:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836547' 2026-02-17T06:41:37.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:37.590 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:37.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705722 2026-02-17T06:41:37.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705722 2026-02-17T06:41:37.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836547 1-38654705722' 2026-02-17T06:41:37.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:37.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:37.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542194 2026-02-17T06:41:37.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542194 2026-02-17T06:41:37.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836547 1-38654705722 2-60129542194' 2026-02-17T06:41:37.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:37.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836547 2026-02-17T06:41:37.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:37.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:37.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836547 2026-02-17T06:41:37.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:37.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836547 2026-02-17T06:41:37.733 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836547 2026-02-17T06:41:37.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836547' 2026-02-17T06:41:37.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:37.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836547 -lt 21474836547 2026-02-17T06:41:37.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:37.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705722 2026-02-17T06:41:37.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:37.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:37.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705722 2026-02-17T06:41:37.885 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:37.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705722 2026-02-17T06:41:37.887 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705722 2026-02-17T06:41:37.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705722' 2026-02-17T06:41:37.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:38.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705722 -lt 38654705722 2026-02-17T06:41:38.042 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:38.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542194 2026-02-17T06:41:38.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:38.044 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:38.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542194 2026-02-17T06:41:38.045 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:38.046 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542194 2026-02-17T06:41:38.046 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542194 2026-02-17T06:41:38.047 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542194' 2026-02-17T06:41:38.047 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:38.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542195 -lt 60129542194 2026-02-17T06:41:38.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:38.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:38.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:38.337 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:38.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:38.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:38.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:38.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:38.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:38.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:38.497 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:38.497 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:38.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:38.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:38.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:38.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836549 2026-02-17T06:41:38.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836549 2026-02-17T06:41:38.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836549' 2026-02-17T06:41:38.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:38.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:38.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705724 2026-02-17T06:41:38.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705724 2026-02-17T06:41:38.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836549 1-38654705724' 2026-02-17T06:41:38.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:38.639 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:38.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542196 2026-02-17T06:41:38.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542196 2026-02-17T06:41:38.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836549 1-38654705724 2-60129542196' 2026-02-17T06:41:38.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:38.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836549 2026-02-17T06:41:38.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:38.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:38.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836549 2026-02-17T06:41:38.712 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:38.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836549 2026-02-17T06:41:38.714 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836549 2026-02-17T06:41:38.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836549' 2026-02-17T06:41:38.714 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:38.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836547 -lt 21474836549 2026-02-17T06:41:38.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:39.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:39.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:40.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836550 -lt 21474836549 2026-02-17T06:41:40.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:40.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705724 2026-02-17T06:41:40.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:40.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:40.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705724 2026-02-17T06:41:40.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:40.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705724 2026-02-17T06:41:40.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705724' 2026-02-17T06:41:40.022 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705724 2026-02-17T06:41:40.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:40.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705725 -lt 38654705724 2026-02-17T06:41:40.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:40.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542196 2026-02-17T06:41:40.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:40.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:40.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542196 2026-02-17T06:41:40.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:40.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542196 2026-02-17T06:41:40.173 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542196 2026-02-17T06:41:40.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542196' 2026-02-17T06:41:40.174 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:40.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542198 -lt 60129542196 2026-02-17T06:41:40.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:40.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:40.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:40.470 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:40.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:40.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:40.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:40.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:40.480 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:40.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:40.626 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:40.627 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:40.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:40.627 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:40.627 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:40.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836552 2026-02-17T06:41:40.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836552 2026-02-17T06:41:40.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836552' 2026-02-17T06:41:40.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:40.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:40.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705727 2026-02-17T06:41:40.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705727 2026-02-17T06:41:40.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836552 1-38654705727' 2026-02-17T06:41:40.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:40.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542200 2026-02-17T06:41:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542200 2026-02-17T06:41:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836552 1-38654705727 2-60129542200' 2026-02-17T06:41:40.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:40.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836552 2026-02-17T06:41:40.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:40.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:40.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836552 2026-02-17T06:41:40.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:40.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836552 2026-02-17T06:41:40.847 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836552 2026-02-17T06:41:40.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836552' 2026-02-17T06:41:40.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:40.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836550 -lt 21474836552 2026-02-17T06:41:40.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:41.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:41.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:42.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836553 -lt 21474836552 2026-02-17T06:41:42.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:42.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705727 2026-02-17T06:41:42.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:42.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:42.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705727 2026-02-17T06:41:42.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:42.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705727 2026-02-17T06:41:42.156 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705727 2026-02-17T06:41:42.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705727' 2026-02-17T06:41:42.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:42.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705728 -lt 38654705727 2026-02-17T06:41:42.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:42.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542200 2026-02-17T06:41:42.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:42.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:42.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542200 2026-02-17T06:41:42.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:42.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542200 2026-02-17T06:41:42.317 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542200 2026-02-17T06:41:42.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542200' 2026-02-17T06:41:42.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:42.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542201 -lt 60129542200 2026-02-17T06:41:42.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:42.479 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:42.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:42.621 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:42.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:42.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:42.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:42.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:42.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:42.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:42.780 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:42.780 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:42.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:42.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:42.780 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:42.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836555 2026-02-17T06:41:42.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836555 2026-02-17T06:41:42.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836555' 2026-02-17T06:41:42.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:42.852 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:42.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705730 2026-02-17T06:41:42.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705730 2026-02-17T06:41:42.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836555 1-38654705730' 2026-02-17T06:41:42.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:42.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:42.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542203 2026-02-17T06:41:42.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542203 2026-02-17T06:41:42.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836555 1-38654705730 2-60129542203' 2026-02-17T06:41:42.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:42.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836555 2026-02-17T06:41:42.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:42.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:42.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836555 2026-02-17T06:41:42.998 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:43.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836555 2026-02-17T06:41:43.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836555' 2026-02-17T06:41:43.000 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836555 2026-02-17T06:41:43.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:43.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836553 -lt 21474836555 2026-02-17T06:41:43.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:44.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:44.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:44.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836556 -lt 21474836555 2026-02-17T06:41:44.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:44.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705730 2026-02-17T06:41:44.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:44.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:44.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705730 2026-02-17T06:41:44.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:44.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705730 2026-02-17T06:41:44.317 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705730' 2026-02-17T06:41:44.317 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705730 2026-02-17T06:41:44.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:44.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705731 -lt 38654705730 2026-02-17T06:41:44.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:44.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542203 2026-02-17T06:41:44.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:44.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:44.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542203 2026-02-17T06:41:44.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:44.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542203 2026-02-17T06:41:44.477 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542203 2026-02-17T06:41:44.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542203' 2026-02-17T06:41:44.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:44.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542204 -lt 60129542203 2026-02-17T06:41:44.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:44.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:44.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:44.777 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:44.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:44.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:44.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:44.787 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:44.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:44.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:44.935 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:44.935 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:44.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:44.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:44.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:45.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836558 2026-02-17T06:41:45.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836558 2026-02-17T06:41:45.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836558' 2026-02-17T06:41:45.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:45.008 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:45.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705734 2026-02-17T06:41:45.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705734 2026-02-17T06:41:45.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836558 1-38654705734' 2026-02-17T06:41:45.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:45.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:45.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542206 2026-02-17T06:41:45.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542206 2026-02-17T06:41:45.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836558 1-38654705734 2-60129542206' 2026-02-17T06:41:45.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:45.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836558 2026-02-17T06:41:45.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:45.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:45.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836558 2026-02-17T06:41:45.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:45.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836558 2026-02-17T06:41:45.151 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836558 2026-02-17T06:41:45.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836558' 2026-02-17T06:41:45.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:45.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836556 -lt 21474836558 2026-02-17T06:41:45.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:46.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:46.309 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:46.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836559 -lt 21474836558 2026-02-17T06:41:46.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:46.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705734 2026-02-17T06:41:46.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:46.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:46.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705734 2026-02-17T06:41:46.465 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:46.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705734 2026-02-17T06:41:46.467 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705734 2026-02-17T06:41:46.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705734' 2026-02-17T06:41:46.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705734 -lt 38654705734 2026-02-17T06:41:46.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:46.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542206 2026-02-17T06:41:46.621 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:46.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:46.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542206 2026-02-17T06:41:46.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:46.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542206 2026-02-17T06:41:46.624 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542206 2026-02-17T06:41:46.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542206' 2026-02-17T06:41:46.625 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:46.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542207 -lt 60129542206 2026-02-17T06:41:46.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:46.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:46.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:46.923 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:46.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:46.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:46.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:46.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:46.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:47.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:47.083 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:47.083 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:47.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:47.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:47.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:47.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836561 2026-02-17T06:41:47.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836561 2026-02-17T06:41:47.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836561' 2026-02-17T06:41:47.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:47.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:47.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705737 2026-02-17T06:41:47.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705737 2026-02-17T06:41:47.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836561 1-38654705737' 2026-02-17T06:41:47.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:47.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:47.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542209 2026-02-17T06:41:47.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542209 2026-02-17T06:41:47.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836561 1-38654705737 2-60129542209' 2026-02-17T06:41:47.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:47.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836561 2026-02-17T06:41:47.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:47.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:47.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836561 2026-02-17T06:41:47.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:47.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836561 2026-02-17T06:41:47.307 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836561 2026-02-17T06:41:47.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836561' 2026-02-17T06:41:47.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:47.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836559 -lt 21474836561 2026-02-17T06:41:47.463 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:48.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:48.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:48.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836562 -lt 21474836561 2026-02-17T06:41:48.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:48.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705737 2026-02-17T06:41:48.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:48.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:48.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705737 2026-02-17T06:41:48.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:48.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705737 2026-02-17T06:41:48.618 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705737 2026-02-17T06:41:48.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705737' 2026-02-17T06:41:48.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:48.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705737 -lt 38654705737 2026-02-17T06:41:48.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:48.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542209 2026-02-17T06:41:48.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:48.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:48.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542209 2026-02-17T06:41:48.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:48.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542209 2026-02-17T06:41:48.777 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542209 2026-02-17T06:41:48.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542209' 2026-02-17T06:41:48.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:48.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542210 -lt 60129542209 2026-02-17T06:41:48.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:48.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:48.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:49.076 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:49.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:49.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:49.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:49.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:49.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:49.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:49.235 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:49.235 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:49.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:49.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:49.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:49.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836564 2026-02-17T06:41:49.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836564 2026-02-17T06:41:49.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836564' 2026-02-17T06:41:49.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:49.311 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:49.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705740 2026-02-17T06:41:49.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705740 2026-02-17T06:41:49.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836564 1-38654705740' 2026-02-17T06:41:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:49.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:49.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542212 2026-02-17T06:41:49.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542212 2026-02-17T06:41:49.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836564 1-38654705740 2-60129542212' 2026-02-17T06:41:49.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:49.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836564 2026-02-17T06:41:49.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:49.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:49.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836564 2026-02-17T06:41:49.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:49.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836564 2026-02-17T06:41:49.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836564' 2026-02-17T06:41:49.459 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836564 2026-02-17T06:41:49.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:49.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836562 -lt 21474836564 2026-02-17T06:41:49.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:50.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:50.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:50.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836565 -lt 21474836564 2026-02-17T06:41:50.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:50.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705740 2026-02-17T06:41:50.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:50.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:50.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705740 2026-02-17T06:41:50.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:50.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705740 2026-02-17T06:41:50.769 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705740 2026-02-17T06:41:50.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705740' 2026-02-17T06:41:50.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:50.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705740 -lt 38654705740 2026-02-17T06:41:50.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:50.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542212 2026-02-17T06:41:50.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:50.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:50.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542212 2026-02-17T06:41:50.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:50.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542212 2026-02-17T06:41:50.937 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542212 2026-02-17T06:41:50.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542212' 2026-02-17T06:41:50.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:51.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542213 -lt 60129542212 2026-02-17T06:41:51.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:51.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:51.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:51.236 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:51.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:51.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:51.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:51.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:51.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:51.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:51.397 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:51.397 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:51.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:51.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:51.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:51.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836567 2026-02-17T06:41:51.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836567 2026-02-17T06:41:51.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836567' 2026-02-17T06:41:51.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:51.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:51.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705743 2026-02-17T06:41:51.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705743 2026-02-17T06:41:51.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836567 1-38654705743' 2026-02-17T06:41:51.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:51.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:51.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542215 2026-02-17T06:41:51.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542215 2026-02-17T06:41:51.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836567 1-38654705743 2-60129542215' 2026-02-17T06:41:51.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:51.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836567 2026-02-17T06:41:51.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:51.612 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:51.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836567 2026-02-17T06:41:51.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:51.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836567 2026-02-17T06:41:51.615 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836567 2026-02-17T06:41:51.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836567' 2026-02-17T06:41:51.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:51.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836565 -lt 21474836567 2026-02-17T06:41:51.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:52.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:52.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:52.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836568 -lt 21474836567 2026-02-17T06:41:52.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:52.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705743 2026-02-17T06:41:52.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:52.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:52.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705743 2026-02-17T06:41:52.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:52.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705743 2026-02-17T06:41:52.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705743' 2026-02-17T06:41:52.922 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705743 2026-02-17T06:41:52.923 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:53.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705743 -lt 38654705743 2026-02-17T06:41:53.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:53.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542215 2026-02-17T06:41:53.081 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:53.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:53.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542215 2026-02-17T06:41:53.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:53.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542215 2026-02-17T06:41:53.085 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542215 2026-02-17T06:41:53.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542215' 2026-02-17T06:41:53.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:53.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542216 -lt 60129542215 2026-02-17T06:41:53.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:53.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:53.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:53.380 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:53.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:53.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:53.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:53.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:53.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:53.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:53.545 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:53.545 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:53.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:53.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:53.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:53.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836571 2026-02-17T06:41:53.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836571 2026-02-17T06:41:53.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836571' 2026-02-17T06:41:53.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:53.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:53.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705746 2026-02-17T06:41:53.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705746 2026-02-17T06:41:53.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836571 1-38654705746' 2026-02-17T06:41:53.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:53.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:53.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542219 2026-02-17T06:41:53.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542219 2026-02-17T06:41:53.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836571 1-38654705746 2-60129542219' 2026-02-17T06:41:53.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:53.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836571 2026-02-17T06:41:53.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:53.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:53.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836571 2026-02-17T06:41:53.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:53.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836571 2026-02-17T06:41:53.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836571' 2026-02-17T06:41:53.760 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836571 2026-02-17T06:41:53.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:53.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836571 -lt 21474836571 2026-02-17T06:41:53.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:53.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705746 2026-02-17T06:41:53.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:53.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:53.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705746 2026-02-17T06:41:53.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:53.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705746 2026-02-17T06:41:53.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705746' 2026-02-17T06:41:53.925 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705746 2026-02-17T06:41:53.925 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:54.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705746 -lt 38654705746 2026-02-17T06:41:54.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:54.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542219 2026-02-17T06:41:54.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:54.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:54.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542219 2026-02-17T06:41:54.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:54.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542219 2026-02-17T06:41:54.086 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542219 2026-02-17T06:41:54.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542219' 2026-02-17T06:41:54.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:54.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542219 -lt 60129542219 2026-02-17T06:41:54.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:54.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:54.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:54.380 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:54.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:54.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:54.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:54.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:54.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:54.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:54.540 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:54.540 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:54.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:54.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:54.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:54.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836573 2026-02-17T06:41:54.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836573 2026-02-17T06:41:54.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836573' 2026-02-17T06:41:54.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:54.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:54.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705748 2026-02-17T06:41:54.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705748 2026-02-17T06:41:54.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836573 1-38654705748' 2026-02-17T06:41:54.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:54.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:54.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542221 2026-02-17T06:41:54.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542221 2026-02-17T06:41:54.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836573 1-38654705748 2-60129542221' 2026-02-17T06:41:54.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:54.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836573 2026-02-17T06:41:54.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:54.757 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:54.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836573 2026-02-17T06:41:54.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:54.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836573 2026-02-17T06:41:54.760 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836573 2026-02-17T06:41:54.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836573' 2026-02-17T06:41:54.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:54.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836571 -lt 21474836573 2026-02-17T06:41:54.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:55.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:55.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:56.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836574 -lt 21474836573 2026-02-17T06:41:56.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:56.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705748 2026-02-17T06:41:56.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:56.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:56.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705748 2026-02-17T06:41:56.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:56.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705748 2026-02-17T06:41:56.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705748' 2026-02-17T06:41:56.077 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705748 2026-02-17T06:41:56.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:56.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705749 -lt 38654705748 2026-02-17T06:41:56.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:56.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542221 2026-02-17T06:41:56.238 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:56.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:56.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542221 2026-02-17T06:41:56.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:56.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542221 2026-02-17T06:41:56.241 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542221 2026-02-17T06:41:56.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542221' 2026-02-17T06:41:56.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:56.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542222 -lt 60129542221 2026-02-17T06:41:56.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:56.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:56.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:56.535 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:56.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:56.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:56.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:56.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:56.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:56.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:56.692 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:56.692 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:56.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:56.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:56.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:56.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836576 2026-02-17T06:41:56.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836576 2026-02-17T06:41:56.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836576' 2026-02-17T06:41:56.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:56.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:56.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705751 2026-02-17T06:41:56.834 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705751 2026-02-17T06:41:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836576 1-38654705751' 2026-02-17T06:41:56.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:56.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:56.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542224 2026-02-17T06:41:56.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542224 2026-02-17T06:41:56.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836576 1-38654705751 2-60129542224' 2026-02-17T06:41:56.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:56.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836576 2026-02-17T06:41:56.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:56.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:56.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836576 2026-02-17T06:41:56.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:56.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836576 2026-02-17T06:41:56.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836576' 2026-02-17T06:41:56.912 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836576 2026-02-17T06:41:56.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:57.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836574 -lt 21474836576 2026-02-17T06:41:57.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:41:58.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:41:58.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:58.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836577 -lt 21474836576 2026-02-17T06:41:58.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:58.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705751 2026-02-17T06:41:58.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:58.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:41:58.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705751 2026-02-17T06:41:58.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:58.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705751 2026-02-17T06:41:58.226 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705751 2026-02-17T06:41:58.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705751' 2026-02-17T06:41:58.227 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:41:58.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705752 -lt 38654705751 2026-02-17T06:41:58.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:58.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542224 2026-02-17T06:41:58.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:58.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:41:58.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542224 2026-02-17T06:41:58.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:58.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542224 2026-02-17T06:41:58.383 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542224 2026-02-17T06:41:58.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542224' 2026-02-17T06:41:58.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:41:58.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542225 -lt 60129542224 2026-02-17T06:41:58.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:41:58.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:41:58.534 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:41:58.674 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:41:58.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:41:58.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:41:58.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:41:58.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:41:58.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:41:58.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:41:58.830 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:41:58.830 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:41:58.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:41:58.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:58.830 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:41:58.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836579 2026-02-17T06:41:58.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836579 2026-02-17T06:41:58.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836579' 2026-02-17T06:41:58.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:58.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:41:58.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705754 2026-02-17T06:41:58.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705754 2026-02-17T06:41:58.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836579 1-38654705754' 2026-02-17T06:41:58.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:41:58.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:41:59.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542227 2026-02-17T06:41:59.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542227 2026-02-17T06:41:59.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836579 1-38654705754 2-60129542227' 2026-02-17T06:41:59.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:41:59.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836579 2026-02-17T06:41:59.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:41:59.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:41:59.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836579 2026-02-17T06:41:59.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:41:59.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836579 2026-02-17T06:41:59.043 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836579 2026-02-17T06:41:59.043 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836579' 2026-02-17T06:41:59.043 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:41:59.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836577 -lt 21474836579 2026-02-17T06:41:59.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:00.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:00.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:00.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836580 -lt 21474836579 2026-02-17T06:42:00.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:00.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705754 2026-02-17T06:42:00.348 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:00.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:00.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705754 2026-02-17T06:42:00.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:00.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705754 2026-02-17T06:42:00.352 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705754 2026-02-17T06:42:00.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705754' 2026-02-17T06:42:00.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:00.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705755 -lt 38654705754 2026-02-17T06:42:00.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:00.515 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542227 2026-02-17T06:42:00.516 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:00.516 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:00.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542227 2026-02-17T06:42:00.517 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:00.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542227 2026-02-17T06:42:00.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542227' 2026-02-17T06:42:00.519 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542227 2026-02-17T06:42:00.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:00.665 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542228 -lt 60129542227 2026-02-17T06:42:00.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:00.666 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:00.667 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:00.810 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:00.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:00.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:00.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:00.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:00.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:00.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:00.970 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:00.970 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:00.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:00.970 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:00.970 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:01.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836582 2026-02-17T06:42:01.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836582 2026-02-17T06:42:01.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836582' 2026-02-17T06:42:01.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:01.040 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:01.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705758 2026-02-17T06:42:01.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705758 2026-02-17T06:42:01.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836582 1-38654705758' 2026-02-17T06:42:01.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:01.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:01.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542230 2026-02-17T06:42:01.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542230 2026-02-17T06:42:01.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836582 1-38654705758 2-60129542230' 2026-02-17T06:42:01.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:01.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836582 2026-02-17T06:42:01.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:01.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:01.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836582 2026-02-17T06:42:01.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:01.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836582 2026-02-17T06:42:01.190 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836582 2026-02-17T06:42:01.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836582' 2026-02-17T06:42:01.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:01.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836580 -lt 21474836582 2026-02-17T06:42:01.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:02.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:02.339 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:02.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836583 -lt 21474836582 2026-02-17T06:42:02.491 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:02.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705758 2026-02-17T06:42:02.492 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:02.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:02.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705758 2026-02-17T06:42:02.494 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:02.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705758 2026-02-17T06:42:02.496 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705758 2026-02-17T06:42:02.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705758' 2026-02-17T06:42:02.496 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:02.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705758 -lt 38654705758 2026-02-17T06:42:02.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:02.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542230 2026-02-17T06:42:02.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:02.658 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:02.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542230 2026-02-17T06:42:02.659 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:02.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542230 2026-02-17T06:42:02.660 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542230 2026-02-17T06:42:02.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542230' 2026-02-17T06:42:02.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:02.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542231 -lt 60129542230 2026-02-17T06:42:02.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:02.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:02.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:02.962 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:02.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:02.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:02.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:02.972 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:02.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:03.118 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:03.119 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:03.119 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:03.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:03.119 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:03.119 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:03.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836585 2026-02-17T06:42:03.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836585 2026-02-17T06:42:03.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836585' 2026-02-17T06:42:03.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:03.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:03.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705761 2026-02-17T06:42:03.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705761 2026-02-17T06:42:03.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836585 1-38654705761' 2026-02-17T06:42:03.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:03.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:03.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542233 2026-02-17T06:42:03.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542233 2026-02-17T06:42:03.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836585 1-38654705761 2-60129542233' 2026-02-17T06:42:03.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:03.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836585 2026-02-17T06:42:03.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:03.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:03.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836585 2026-02-17T06:42:03.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:03.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836585 2026-02-17T06:42:03.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836585' 2026-02-17T06:42:03.335 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836585 2026-02-17T06:42:03.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:03.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836583 -lt 21474836585 2026-02-17T06:42:03.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:04.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:04.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:04.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836586 -lt 21474836585 2026-02-17T06:42:04.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:04.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705761 2026-02-17T06:42:04.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:04.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:04.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705761 2026-02-17T06:42:04.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:04.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705761 2026-02-17T06:42:04.640 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705761 2026-02-17T06:42:04.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705761' 2026-02-17T06:42:04.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705761 -lt 38654705761 2026-02-17T06:42:04.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:04.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542233 2026-02-17T06:42:04.798 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:04.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:04.799 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542233 2026-02-17T06:42:04.800 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:04.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542233 2026-02-17T06:42:04.801 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542233 2026-02-17T06:42:04.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542233' 2026-02-17T06:42:04.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:04.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542234 -lt 60129542233 2026-02-17T06:42:04.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:04.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:04.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:05.100 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:05.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:05.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:05.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:05.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:05.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:05.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:05.255 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:05.255 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:05.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:05.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:05.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:05.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836588 2026-02-17T06:42:05.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836588 2026-02-17T06:42:05.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836588' 2026-02-17T06:42:05.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:05.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:05.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705764 2026-02-17T06:42:05.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705764 2026-02-17T06:42:05.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836588 1-38654705764' 2026-02-17T06:42:05.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:05.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:05.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542236 2026-02-17T06:42:05.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542236 2026-02-17T06:42:05.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836588 1-38654705764 2-60129542236' 2026-02-17T06:42:05.469 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:05.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836588 2026-02-17T06:42:05.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:05.470 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:05.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836588 2026-02-17T06:42:05.471 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:05.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836588 2026-02-17T06:42:05.473 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836588 2026-02-17T06:42:05.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836588' 2026-02-17T06:42:05.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:05.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836586 -lt 21474836588 2026-02-17T06:42:05.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:06.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:06.632 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:06.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836589 -lt 21474836588 2026-02-17T06:42:06.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:06.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705764 2026-02-17T06:42:06.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:06.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:06.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705764 2026-02-17T06:42:06.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:06.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705764 2026-02-17T06:42:06.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705764' 2026-02-17T06:42:06.793 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705764 2026-02-17T06:42:06.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:06.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705764 -lt 38654705764 2026-02-17T06:42:06.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:06.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542236 2026-02-17T06:42:06.951 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:06.952 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:06.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542236 2026-02-17T06:42:06.952 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:06.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542236 2026-02-17T06:42:06.954 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542236 2026-02-17T06:42:06.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542236' 2026-02-17T06:42:06.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:07.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542237 -lt 60129542236 2026-02-17T06:42:07.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:07.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:07.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:07.248 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:07.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:07.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:07.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:07.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:07.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:07.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:07.404 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:07.404 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:07.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:07.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:07.404 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:07.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836591 2026-02-17T06:42:07.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836591 2026-02-17T06:42:07.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836591' 2026-02-17T06:42:07.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:07.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705767 2026-02-17T06:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705767 2026-02-17T06:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836591 1-38654705767' 2026-02-17T06:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:07.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:07.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542239 2026-02-17T06:42:07.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542239 2026-02-17T06:42:07.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836591 1-38654705767 2-60129542239' 2026-02-17T06:42:07.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:07.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836591 2026-02-17T06:42:07.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:07.621 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:07.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836591 2026-02-17T06:42:07.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:07.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836591 2026-02-17T06:42:07.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836591' 2026-02-17T06:42:07.623 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836591 2026-02-17T06:42:07.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:07.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836589 -lt 21474836591 2026-02-17T06:42:07.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:08.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:08.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:08.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836592 -lt 21474836591 2026-02-17T06:42:08.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:08.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705767 2026-02-17T06:42:08.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:08.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:08.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705767 2026-02-17T06:42:08.932 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:08.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705767 2026-02-17T06:42:08.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705767' 2026-02-17T06:42:08.935 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705767 2026-02-17T06:42:08.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:09.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705767 -lt 38654705767 2026-02-17T06:42:09.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:09.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542239 2026-02-17T06:42:09.083 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:09.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:09.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542239 2026-02-17T06:42:09.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:09.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542239 2026-02-17T06:42:09.087 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542239 2026-02-17T06:42:09.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542239' 2026-02-17T06:42:09.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:09.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542240 -lt 60129542239 2026-02-17T06:42:09.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:09.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:09.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:09.388 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:09.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:09.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:09.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:09.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:09.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:09.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:09.547 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:09.547 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:09.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:09.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:09.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:09.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836595 2026-02-17T06:42:09.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836595 2026-02-17T06:42:09.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595' 2026-02-17T06:42:09.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:09.618 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:09.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705770 2026-02-17T06:42:09.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705770 2026-02-17T06:42:09.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595 1-38654705770' 2026-02-17T06:42:09.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:09.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:09.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542243 2026-02-17T06:42:09.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542243 2026-02-17T06:42:09.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595 1-38654705770 2-60129542243' 2026-02-17T06:42:09.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:09.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836595 2026-02-17T06:42:09.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:09.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:09.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836595 2026-02-17T06:42:09.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:09.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836595 2026-02-17T06:42:09.766 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836595 2026-02-17T06:42:09.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836595' 2026-02-17T06:42:09.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:09.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836595 -lt 21474836595 2026-02-17T06:42:09.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:09.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705770 2026-02-17T06:42:09.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:09.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:09.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705770 2026-02-17T06:42:09.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:09.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705770 2026-02-17T06:42:09.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705770' 2026-02-17T06:42:09.933 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705770 2026-02-17T06:42:09.934 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:10.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705770 -lt 38654705770 2026-02-17T06:42:10.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:10.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542243 2026-02-17T06:42:10.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:10.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:10.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542243 2026-02-17T06:42:10.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:10.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542243 2026-02-17T06:42:10.097 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542243 2026-02-17T06:42:10.097 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542243' 2026-02-17T06:42:10.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:10.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542243 -lt 60129542243 2026-02-17T06:42:10.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:10.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:10.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:10.395 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:10.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:10.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:10.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:10.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:10.404 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:10.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:10.549 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:10.549 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:10.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:10.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:10.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:10.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836597 2026-02-17T06:42:10.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836597 2026-02-17T06:42:10.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836597' 2026-02-17T06:42:10.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:10.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:10.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705772 2026-02-17T06:42:10.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705772 2026-02-17T06:42:10.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836597 1-38654705772' 2026-02-17T06:42:10.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:10.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:10.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542245 2026-02-17T06:42:10.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542245 2026-02-17T06:42:10.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836597 1-38654705772 2-60129542245' 2026-02-17T06:42:10.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:10.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836597 2026-02-17T06:42:10.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:10.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:10.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836597 2026-02-17T06:42:10.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:10.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836597 2026-02-17T06:42:10.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836597' 2026-02-17T06:42:10.771 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836597 2026-02-17T06:42:10.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:10.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836595 -lt 21474836597 2026-02-17T06:42:10.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:11.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:11.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836598 -lt 21474836597 2026-02-17T06:42:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:12.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705772 2026-02-17T06:42:12.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:12.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:12.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705772 2026-02-17T06:42:12.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:12.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705772 2026-02-17T06:42:12.079 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705772 2026-02-17T06:42:12.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705772' 2026-02-17T06:42:12.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:12.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705773 -lt 38654705772 2026-02-17T06:42:12.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:12.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542245 2026-02-17T06:42:12.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:12.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:12.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542245 2026-02-17T06:42:12.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:12.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542245 2026-02-17T06:42:12.242 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542245' 2026-02-17T06:42:12.243 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542245 2026-02-17T06:42:12.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542246 -lt 60129542245 2026-02-17T06:42:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:12.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:12.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:12.529 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:12.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:12.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:12.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:12.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:12.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:12.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:12.690 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:12.690 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:12.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:12.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:12.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836600 2026-02-17T06:42:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836600 2026-02-17T06:42:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836600' 2026-02-17T06:42:12.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:12.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:12.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705775 2026-02-17T06:42:12.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705775 2026-02-17T06:42:12.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836600 1-38654705775' 2026-02-17T06:42:12.835 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:12.836 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:12.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542248 2026-02-17T06:42:12.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542248 2026-02-17T06:42:12.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836600 1-38654705775 2-60129542248' 2026-02-17T06:42:12.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:12.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836600 2026-02-17T06:42:12.908 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:12.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:12.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836600 2026-02-17T06:42:12.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:12.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836600 2026-02-17T06:42:12.911 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836600 2026-02-17T06:42:12.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836600' 2026-02-17T06:42:12.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:13.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836598 -lt 21474836600 2026-02-17T06:42:13.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:14.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:14.069 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:14.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836601 -lt 21474836600 2026-02-17T06:42:14.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:14.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705775 2026-02-17T06:42:14.222 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:14.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:14.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705775 2026-02-17T06:42:14.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:14.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705775 2026-02-17T06:42:14.226 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705775 2026-02-17T06:42:14.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705775' 2026-02-17T06:42:14.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:14.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705776 -lt 38654705775 2026-02-17T06:42:14.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:14.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542248 2026-02-17T06:42:14.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:14.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:14.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542248 2026-02-17T06:42:14.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:14.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542248 2026-02-17T06:42:14.382 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542248 2026-02-17T06:42:14.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542248' 2026-02-17T06:42:14.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:14.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542249 -lt 60129542248 2026-02-17T06:42:14.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:14.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:14.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:14.683 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:14.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:14.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:14.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:14.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:14.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:14.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:14.846 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:14.846 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:14.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:14.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:14.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:14.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836603 2026-02-17T06:42:14.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836603 2026-02-17T06:42:14.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836603' 2026-02-17T06:42:14.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:14.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:14.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705778 2026-02-17T06:42:14.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705778 2026-02-17T06:42:14.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836603 1-38654705778' 2026-02-17T06:42:14.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:14.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:15.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542251 2026-02-17T06:42:15.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542251 2026-02-17T06:42:15.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836603 1-38654705778 2-60129542251' 2026-02-17T06:42:15.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:15.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836603 2026-02-17T06:42:15.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:15.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:15.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836603 2026-02-17T06:42:15.065 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:15.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836603 2026-02-17T06:42:15.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836603' 2026-02-17T06:42:15.067 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836603 2026-02-17T06:42:15.067 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:15.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836601 -lt 21474836603 2026-02-17T06:42:15.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:16.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:16.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:16.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836604 -lt 21474836603 2026-02-17T06:42:16.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:16.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705778 2026-02-17T06:42:16.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:16.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:16.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705778 2026-02-17T06:42:16.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:16.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705778 2026-02-17T06:42:16.381 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705778 2026-02-17T06:42:16.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705778' 2026-02-17T06:42:16.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:16.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705779 -lt 38654705778 2026-02-17T06:42:16.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:16.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542251 2026-02-17T06:42:16.539 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:16.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:16.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542251 2026-02-17T06:42:16.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542251 2026-02-17T06:42:16.542 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542251 2026-02-17T06:42:16.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542251' 2026-02-17T06:42:16.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:16.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542252 -lt 60129542251 2026-02-17T06:42:16.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:16.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:16.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:16.833 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:16.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:16.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:16.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:16.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:16.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:16.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:16.993 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:16.993 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:16.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:16.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:16.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:17.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836606 2026-02-17T06:42:17.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836606 2026-02-17T06:42:17.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836606' 2026-02-17T06:42:17.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:17.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:17.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705782 2026-02-17T06:42:17.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705782 2026-02-17T06:42:17.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836606 1-38654705782' 2026-02-17T06:42:17.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:17.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:17.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542254 2026-02-17T06:42:17.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542254 2026-02-17T06:42:17.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836606 1-38654705782 2-60129542254' 2026-02-17T06:42:17.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:17.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836606 2026-02-17T06:42:17.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:17.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:17.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836606 2026-02-17T06:42:17.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:17.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836606 2026-02-17T06:42:17.212 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836606 2026-02-17T06:42:17.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836606' 2026-02-17T06:42:17.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:17.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836604 -lt 21474836606 2026-02-17T06:42:17.370 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:18.371 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:18.371 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:18.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836607 -lt 21474836606 2026-02-17T06:42:18.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:18.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705782 2026-02-17T06:42:18.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:18.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:18.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705782 2026-02-17T06:42:18.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:18.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705782 2026-02-17T06:42:18.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705782' 2026-02-17T06:42:18.528 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705782 2026-02-17T06:42:18.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:18.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705782 -lt 38654705782 2026-02-17T06:42:18.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:18.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542254 2026-02-17T06:42:18.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:18.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:18.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542254 2026-02-17T06:42:18.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:18.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542254 2026-02-17T06:42:18.695 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542254 2026-02-17T06:42:18.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542254' 2026-02-17T06:42:18.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:18.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542255 -lt 60129542254 2026-02-17T06:42:18.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:18.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:18.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:18.988 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:18.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:18.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:18.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:18.997 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:18.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:19.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:19.148 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:19.148 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:19.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:19.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:19.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:19.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836609 2026-02-17T06:42:19.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836609 2026-02-17T06:42:19.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609' 2026-02-17T06:42:19.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:19.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:19.292 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705785 2026-02-17T06:42:19.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705785 2026-02-17T06:42:19.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609 1-38654705785' 2026-02-17T06:42:19.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:19.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:19.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542257 2026-02-17T06:42:19.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542257 2026-02-17T06:42:19.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609 1-38654705785 2-60129542257' 2026-02-17T06:42:19.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:19.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836609 2026-02-17T06:42:19.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:19.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:19.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836609 2026-02-17T06:42:19.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:19.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836609 2026-02-17T06:42:19.367 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836609' 2026-02-17T06:42:19.367 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836609 2026-02-17T06:42:19.368 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:19.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836607 -lt 21474836609 2026-02-17T06:42:19.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:20.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:20.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:20.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836610 -lt 21474836609 2026-02-17T06:42:20.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:20.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705785 2026-02-17T06:42:20.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:20.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:20.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705785 2026-02-17T06:42:20.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:20.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705785 2026-02-17T06:42:20.687 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705785 2026-02-17T06:42:20.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705785' 2026-02-17T06:42:20.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:20.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705785 -lt 38654705785 2026-02-17T06:42:20.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:20.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542257 2026-02-17T06:42:20.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:20.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:20.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542257 2026-02-17T06:42:20.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:20.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542257 2026-02-17T06:42:20.843 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542257 2026-02-17T06:42:20.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542257' 2026-02-17T06:42:20.843 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:21.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542258 -lt 60129542257 2026-02-17T06:42:21.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:21.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:21.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:21.145 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:21.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:21.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:21.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:21.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:21.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:21.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:21.303 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:21.303 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:21.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:21.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:21.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:21.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836612 2026-02-17T06:42:21.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836612 2026-02-17T06:42:21.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612' 2026-02-17T06:42:21.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:21.376 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:21.446 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705788 2026-02-17T06:42:21.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705788 2026-02-17T06:42:21.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612 1-38654705788' 2026-02-17T06:42:21.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:21.447 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:21.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542260 2026-02-17T06:42:21.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542260 2026-02-17T06:42:21.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612 1-38654705788 2-60129542260' 2026-02-17T06:42:21.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:21.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836612 2026-02-17T06:42:21.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:21.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:21.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836612 2026-02-17T06:42:21.520 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:21.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836612 2026-02-17T06:42:21.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836612' 2026-02-17T06:42:21.522 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836612 2026-02-17T06:42:21.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:21.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836610 -lt 21474836612 2026-02-17T06:42:21.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:22.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:22.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:22.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836613 -lt 21474836612 2026-02-17T06:42:22.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:22.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705788 2026-02-17T06:42:22.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:22.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:22.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705788 2026-02-17T06:42:22.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:22.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705788 2026-02-17T06:42:22.826 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705788 2026-02-17T06:42:22.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705788' 2026-02-17T06:42:22.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:22.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705788 -lt 38654705788 2026-02-17T06:42:22.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:22.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542260 2026-02-17T06:42:22.991 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:22.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:22.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542260 2026-02-17T06:42:22.992 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:22.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542260 2026-02-17T06:42:22.994 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542260 2026-02-17T06:42:22.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542260' 2026-02-17T06:42:22.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:23.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542261 -lt 60129542260 2026-02-17T06:42:23.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:23.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:23.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:23.288 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:23.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:23.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:23.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:23.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:23.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:23.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:23.449 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:23.449 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:23.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:23.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:23.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:23.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836615 2026-02-17T06:42:23.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836615 2026-02-17T06:42:23.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836615' 2026-02-17T06:42:23.519 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:23.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:23.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705791 2026-02-17T06:42:23.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705791 2026-02-17T06:42:23.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836615 1-38654705791' 2026-02-17T06:42:23.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:23.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:23.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542263 2026-02-17T06:42:23.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542263 2026-02-17T06:42:23.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836615 1-38654705791 2-60129542263' 2026-02-17T06:42:23.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:23.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836615 2026-02-17T06:42:23.661 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:23.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:23.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836615 2026-02-17T06:42:23.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:23.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836615 2026-02-17T06:42:23.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836615' 2026-02-17T06:42:23.664 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836615 2026-02-17T06:42:23.664 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:23.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836613 -lt 21474836615 2026-02-17T06:42:23.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:24.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:24.815 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:24.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836616 -lt 21474836615 2026-02-17T06:42:24.969 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:24.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705791 2026-02-17T06:42:24.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:24.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:24.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705791 2026-02-17T06:42:24.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:24.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705791 2026-02-17T06:42:24.973 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705791 2026-02-17T06:42:24.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705791' 2026-02-17T06:42:24.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:25.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705791 -lt 38654705791 2026-02-17T06:42:25.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:25.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542263 2026-02-17T06:42:25.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:25.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:25.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542263 2026-02-17T06:42:25.127 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:25.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542263 2026-02-17T06:42:25.128 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542263' 2026-02-17T06:42:25.128 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542263 2026-02-17T06:42:25.129 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:25.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542264 -lt 60129542263 2026-02-17T06:42:25.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:25.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:25.280 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:25.419 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:25.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:25.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:25.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:25.428 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:25.428 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:25.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:25.577 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:25.577 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:25.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:25.578 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:25.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:25.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836619 2026-02-17T06:42:25.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836619 2026-02-17T06:42:25.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836619' 2026-02-17T06:42:25.647 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:25.647 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:25.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705794 2026-02-17T06:42:25.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705794 2026-02-17T06:42:25.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836619 1-38654705794' 2026-02-17T06:42:25.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:25.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:25.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542267 2026-02-17T06:42:25.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542267 2026-02-17T06:42:25.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836619 1-38654705794 2-60129542267' 2026-02-17T06:42:25.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:25.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836619 2026-02-17T06:42:25.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:25.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:25.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836619 2026-02-17T06:42:25.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:25.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836619 2026-02-17T06:42:25.790 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836619 2026-02-17T06:42:25.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836619' 2026-02-17T06:42:25.791 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:25.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836619 -lt 21474836619 2026-02-17T06:42:25.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:25.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705794 2026-02-17T06:42:25.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:25.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:25.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705794 2026-02-17T06:42:25.940 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:25.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705794 2026-02-17T06:42:25.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705794' 2026-02-17T06:42:25.942 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705794 2026-02-17T06:42:25.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:26.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705794 -lt 38654705794 2026-02-17T06:42:26.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:26.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542267 2026-02-17T06:42:26.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:26.105 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:26.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542267 2026-02-17T06:42:26.106 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:26.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542267 2026-02-17T06:42:26.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542267' 2026-02-17T06:42:26.107 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542267 2026-02-17T06:42:26.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:26.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542266 -lt 60129542267 2026-02-17T06:42:26.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:27.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:27.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:27.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542266 -lt 60129542267 2026-02-17T06:42:27.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:28.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T06:42:28.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:28.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542269 -lt 60129542267 2026-02-17T06:42:28.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:28.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:28.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:28.718 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:28.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:28.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:28.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:28.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:28.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:28.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:28.883 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:28.883 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:28.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:28.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:28.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:28.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836623 2026-02-17T06:42:28.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836623 2026-02-17T06:42:28.954 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836623' 2026-02-17T06:42:28.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:28.955 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:29.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705798 2026-02-17T06:42:29.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705798 2026-02-17T06:42:29.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836623 1-38654705798' 2026-02-17T06:42:29.027 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:29.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:29.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542271 2026-02-17T06:42:29.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542271 2026-02-17T06:42:29.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836623 1-38654705798 2-60129542271' 2026-02-17T06:42:29.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:29.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836623 2026-02-17T06:42:29.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:29.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:29.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836623 2026-02-17T06:42:29.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:29.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836623 2026-02-17T06:42:29.102 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836623 2026-02-17T06:42:29.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836623' 2026-02-17T06:42:29.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:29.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836621 -lt 21474836623 2026-02-17T06:42:29.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:30.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:30.257 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:30.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836624 -lt 21474836623 2026-02-17T06:42:30.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:30.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705798 2026-02-17T06:42:30.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:30.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:30.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705798 2026-02-17T06:42:30.409 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:30.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705798 2026-02-17T06:42:30.410 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705798 2026-02-17T06:42:30.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705798' 2026-02-17T06:42:30.411 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:30.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705799 -lt 38654705798 2026-02-17T06:42:30.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:30.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542271 2026-02-17T06:42:30.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:30.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:30.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542271 2026-02-17T06:42:30.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:30.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542271 2026-02-17T06:42:30.577 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542271 2026-02-17T06:42:30.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542271' 2026-02-17T06:42:30.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:30.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542272 -lt 60129542271 2026-02-17T06:42:30.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:30.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:30.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:30.877 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:30.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:30.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:30.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:30.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:30.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:31.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:31.037 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:31.037 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:31.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:31.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:31.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:31.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836626 2026-02-17T06:42:31.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836626 2026-02-17T06:42:31.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836626' 2026-02-17T06:42:31.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:31.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:31.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705802 2026-02-17T06:42:31.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705802 2026-02-17T06:42:31.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836626 1-38654705802' 2026-02-17T06:42:31.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:31.181 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:31.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542274 2026-02-17T06:42:31.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542274 2026-02-17T06:42:31.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836626 1-38654705802 2-60129542274' 2026-02-17T06:42:31.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:31.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836626 2026-02-17T06:42:31.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:31.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:31.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836626 2026-02-17T06:42:31.254 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:31.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836626 2026-02-17T06:42:31.255 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836626 2026-02-17T06:42:31.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836626' 2026-02-17T06:42:31.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:31.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836624 -lt 21474836626 2026-02-17T06:42:31.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:32.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:32.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:32.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836627 -lt 21474836626 2026-02-17T06:42:32.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:32.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705802 2026-02-17T06:42:32.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:32.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:32.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705802 2026-02-17T06:42:32.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:32.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705802 2026-02-17T06:42:32.575 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705802 2026-02-17T06:42:32.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705802' 2026-02-17T06:42:32.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:32.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705802 -lt 38654705802 2026-02-17T06:42:32.725 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:32.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542274 2026-02-17T06:42:32.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:32.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:32.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542274 2026-02-17T06:42:32.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:32.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542274 2026-02-17T06:42:32.729 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542274 2026-02-17T06:42:32.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542274' 2026-02-17T06:42:32.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:32.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542275 -lt 60129542274 2026-02-17T06:42:32.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:32.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:32.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:33.031 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:33.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:33.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:33.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:33.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:33.042 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:33.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:33.192 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:33.192 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:33.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:33.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:33.192 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:33.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836629 2026-02-17T06:42:33.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836629 2026-02-17T06:42:33.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836629' 2026-02-17T06:42:33.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:33.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:33.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705805 2026-02-17T06:42:33.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705805 2026-02-17T06:42:33.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836629 1-38654705805' 2026-02-17T06:42:33.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:33.340 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:33.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542277 2026-02-17T06:42:33.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542277 2026-02-17T06:42:33.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836629 1-38654705805 2-60129542277' 2026-02-17T06:42:33.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:33.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836629 2026-02-17T06:42:33.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:33.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:33.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836629 2026-02-17T06:42:33.414 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:33.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836629 2026-02-17T06:42:33.415 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836629 2026-02-17T06:42:33.415 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836629' 2026-02-17T06:42:33.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:33.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836627 -lt 21474836629 2026-02-17T06:42:33.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:34.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:34.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:34.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836630 -lt 21474836629 2026-02-17T06:42:34.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:34.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705805 2026-02-17T06:42:34.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:34.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:34.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705805 2026-02-17T06:42:34.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:34.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705805 2026-02-17T06:42:34.733 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705805 2026-02-17T06:42:34.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705805' 2026-02-17T06:42:34.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:34.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705805 -lt 38654705805 2026-02-17T06:42:34.893 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:34.893 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542277 2026-02-17T06:42:34.893 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:34.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:34.895 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542277 2026-02-17T06:42:34.895 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:34.896 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542277 2026-02-17T06:42:34.897 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542277 2026-02-17T06:42:34.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542277' 2026-02-17T06:42:34.897 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:35.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542278 -lt 60129542277 2026-02-17T06:42:35.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:35.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:35.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:35.193 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:35.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:35.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:35.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:35.350 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:35.351 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:35.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:35.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:35.351 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:35.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836632 2026-02-17T06:42:35.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836632 2026-02-17T06:42:35.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836632' 2026-02-17T06:42:35.424 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:35.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:35.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705808 2026-02-17T06:42:35.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705808 2026-02-17T06:42:35.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836632 1-38654705808' 2026-02-17T06:42:35.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:35.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:35.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542280 2026-02-17T06:42:35.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542280 2026-02-17T06:42:35.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836632 1-38654705808 2-60129542280' 2026-02-17T06:42:35.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:35.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836632 2026-02-17T06:42:35.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:35.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:35.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836632 2026-02-17T06:42:35.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:35.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836632 2026-02-17T06:42:35.572 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836632 2026-02-17T06:42:35.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836632' 2026-02-17T06:42:35.573 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:35.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836630 -lt 21474836632 2026-02-17T06:42:35.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:36.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:36.736 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:36.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836633 -lt 21474836632 2026-02-17T06:42:36.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:36.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705808 2026-02-17T06:42:36.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:36.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:36.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705808 2026-02-17T06:42:36.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:36.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705808 2026-02-17T06:42:36.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705808' 2026-02-17T06:42:36.892 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705808 2026-02-17T06:42:36.893 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:37.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705808 -lt 38654705808 2026-02-17T06:42:37.054 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:37.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542280 2026-02-17T06:42:37.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:37.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:37.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542280 2026-02-17T06:42:37.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:37.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542280 2026-02-17T06:42:37.058 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542280 2026-02-17T06:42:37.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542280' 2026-02-17T06:42:37.059 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:37.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542281 -lt 60129542280 2026-02-17T06:42:37.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:37.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:37.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:37.357 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:37.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:37.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:37.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:37.512 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:37.512 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:37.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:37.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:37.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:37.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836636 2026-02-17T06:42:37.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836636 2026-02-17T06:42:37.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836636' 2026-02-17T06:42:37.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:37.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:37.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705811 2026-02-17T06:42:37.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705811 2026-02-17T06:42:37.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836636 1-38654705811' 2026-02-17T06:42:37.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:37.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:37.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542283 2026-02-17T06:42:37.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542283 2026-02-17T06:42:37.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836636 1-38654705811 2-60129542283' 2026-02-17T06:42:37.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:37.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836636 2026-02-17T06:42:37.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:37.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:37.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836636 2026-02-17T06:42:37.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:37.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836636 2026-02-17T06:42:37.730 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836636 2026-02-17T06:42:37.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836636' 2026-02-17T06:42:37.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:37.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836636 -lt 21474836636 2026-02-17T06:42:37.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:37.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705811 2026-02-17T06:42:37.879 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:37.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:37.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705811 2026-02-17T06:42:37.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:37.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705811 2026-02-17T06:42:37.883 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705811 2026-02-17T06:42:37.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705811' 2026-02-17T06:42:37.883 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:38.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705811 -lt 38654705811 2026-02-17T06:42:38.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:38.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542283 2026-02-17T06:42:38.035 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:38.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:38.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542283 2026-02-17T06:42:38.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:38.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542283 2026-02-17T06:42:38.039 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542283 2026-02-17T06:42:38.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542283' 2026-02-17T06:42:38.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:38.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542284 -lt 60129542283 2026-02-17T06:42:38.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:38.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:38.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:38.342 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:38.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:38.352 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:38.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:38.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:38.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:38.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:38.500 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:38.500 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:38.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:38.500 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:38.500 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:38.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836638 2026-02-17T06:42:38.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836638 2026-02-17T06:42:38.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836638' 2026-02-17T06:42:38.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:38.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:38.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705813 2026-02-17T06:42:38.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705813 2026-02-17T06:42:38.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836638 1-38654705813' 2026-02-17T06:42:38.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:38.643 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:38.714 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542285 2026-02-17T06:42:38.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542285 2026-02-17T06:42:38.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836638 1-38654705813 2-60129542285' 2026-02-17T06:42:38.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:38.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836638 2026-02-17T06:42:38.715 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:38.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:38.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836638 2026-02-17T06:42:38.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:38.718 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836638 2026-02-17T06:42:38.718 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836638 2026-02-17T06:42:38.719 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836638' 2026-02-17T06:42:38.719 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:38.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836636 -lt 21474836638 2026-02-17T06:42:38.867 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:39.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:39.868 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:40.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836639 -lt 21474836638 2026-02-17T06:42:40.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:40.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705813 2026-02-17T06:42:40.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:40.024 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:40.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705813 2026-02-17T06:42:40.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:40.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705813 2026-02-17T06:42:40.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705813' 2026-02-17T06:42:40.027 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705813 2026-02-17T06:42:40.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:40.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705814 -lt 38654705813 2026-02-17T06:42:40.185 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:40.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542285 2026-02-17T06:42:40.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:40.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:40.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542285 2026-02-17T06:42:40.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:40.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542285 2026-02-17T06:42:40.190 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542285 2026-02-17T06:42:40.190 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542285' 2026-02-17T06:42:40.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:40.339 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542287 -lt 60129542285 2026-02-17T06:42:40.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:40.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:40.340 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:40.479 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:40.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:40.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:40.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:40.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:40.489 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:40.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:40.640 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:40.640 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:40.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:40.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:40.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:40.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836641 2026-02-17T06:42:40.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836641 2026-02-17T06:42:40.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836641' 2026-02-17T06:42:40.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:40.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:40.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705816 2026-02-17T06:42:40.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705816 2026-02-17T06:42:40.783 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836641 1-38654705816' 2026-02-17T06:42:40.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:40.784 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:40.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542289 2026-02-17T06:42:40.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542289 2026-02-17T06:42:40.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836641 1-38654705816 2-60129542289' 2026-02-17T06:42:40.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:40.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836641 2026-02-17T06:42:40.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:40.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:40.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836641 2026-02-17T06:42:40.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:40.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836641 2026-02-17T06:42:40.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836641' 2026-02-17T06:42:40.862 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836641 2026-02-17T06:42:40.863 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:41.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836639 -lt 21474836641 2026-02-17T06:42:41.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:42.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:42.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:42.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836642 -lt 21474836641 2026-02-17T06:42:42.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:42.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705816 2026-02-17T06:42:42.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:42.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:42.169 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705816 2026-02-17T06:42:42.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:42.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705816 2026-02-17T06:42:42.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705816' 2026-02-17T06:42:42.171 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705816 2026-02-17T06:42:42.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:42.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705817 -lt 38654705816 2026-02-17T06:42:42.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:42.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542289 2026-02-17T06:42:42.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:42.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:42.333 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542289 2026-02-17T06:42:42.334 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:42.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542289 2026-02-17T06:42:42.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542289' 2026-02-17T06:42:42.335 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542289 2026-02-17T06:42:42.336 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:42.487 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542290 -lt 60129542289 2026-02-17T06:42:42.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:42.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:42.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:42.624 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:42.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:42.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:42.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:42.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:42.634 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:42.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:42.786 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:42.786 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:42.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:42.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:42.786 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:42.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836644 2026-02-17T06:42:42.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836644 2026-02-17T06:42:42.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836644' 2026-02-17T06:42:42.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:42.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:42.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705819 2026-02-17T06:42:42.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705819 2026-02-17T06:42:42.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836644 1-38654705819' 2026-02-17T06:42:42.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:42.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:43.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542292 2026-02-17T06:42:43.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542292 2026-02-17T06:42:43.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836644 1-38654705819 2-60129542292' 2026-02-17T06:42:43.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:43.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836644 2026-02-17T06:42:43.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:43.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:43.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836644 2026-02-17T06:42:43.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:43.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836644 2026-02-17T06:42:43.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836644' 2026-02-17T06:42:43.005 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836644 2026-02-17T06:42:43.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:43.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836642 -lt 21474836644 2026-02-17T06:42:43.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:44.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:44.165 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:44.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836645 -lt 21474836644 2026-02-17T06:42:44.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:44.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705819 2026-02-17T06:42:44.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:44.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:44.318 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705819 2026-02-17T06:42:44.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:44.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705819 2026-02-17T06:42:44.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705819' 2026-02-17T06:42:44.320 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705819 2026-02-17T06:42:44.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:44.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705820 -lt 38654705819 2026-02-17T06:42:44.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:44.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542292 2026-02-17T06:42:44.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:44.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:44.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542292 2026-02-17T06:42:44.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:44.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542292 2026-02-17T06:42:44.485 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542292 2026-02-17T06:42:44.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542292' 2026-02-17T06:42:44.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:44.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542293 -lt 60129542292 2026-02-17T06:42:44.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:44.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:44.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:44.779 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:44.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:44.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:44.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:44.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:44.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:44.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:44.937 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:44.937 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:44.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:44.938 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:44.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:45.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836647 2026-02-17T06:42:45.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836647 2026-02-17T06:42:45.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836647' 2026-02-17T06:42:45.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:45.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:45.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705823 2026-02-17T06:42:45.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705823 2026-02-17T06:42:45.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836647 1-38654705823' 2026-02-17T06:42:45.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:45.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:45.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542295 2026-02-17T06:42:45.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542295 2026-02-17T06:42:45.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836647 1-38654705823 2-60129542295' 2026-02-17T06:42:45.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:45.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836647 2026-02-17T06:42:45.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:45.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:45.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836647 2026-02-17T06:42:45.149 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:45.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836647 2026-02-17T06:42:45.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836647' 2026-02-17T06:42:45.151 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836647 2026-02-17T06:42:45.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:45.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836645 -lt 21474836647 2026-02-17T06:42:45.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:46.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:46.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:46.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836648 -lt 21474836647 2026-02-17T06:42:46.449 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:46.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705823 2026-02-17T06:42:46.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:46.451 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:46.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705823 2026-02-17T06:42:46.452 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:46.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705823 2026-02-17T06:42:46.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705823' 2026-02-17T06:42:46.454 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705823 2026-02-17T06:42:46.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:46.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705823 -lt 38654705823 2026-02-17T06:42:46.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:46.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542295 2026-02-17T06:42:46.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:46.611 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:46.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542295 2026-02-17T06:42:46.612 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:46.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542295 2026-02-17T06:42:46.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542295' 2026-02-17T06:42:46.614 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542295 2026-02-17T06:42:46.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:46.765 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542296 -lt 60129542295 2026-02-17T06:42:46.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:46.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:46.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:46.907 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:46.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:46.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:46.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:46.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:46.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:47.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:47.070 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:47.070 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:47.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:47.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:47.070 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:47.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836650 2026-02-17T06:42:47.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836650 2026-02-17T06:42:47.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836650' 2026-02-17T06:42:47.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:47.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:47.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705826 2026-02-17T06:42:47.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705826 2026-02-17T06:42:47.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836650 1-38654705826' 2026-02-17T06:42:47.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:47.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:47.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542298 2026-02-17T06:42:47.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542298 2026-02-17T06:42:47.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836650 1-38654705826 2-60129542298' 2026-02-17T06:42:47.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:47.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836650 2026-02-17T06:42:47.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:47.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:47.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836650 2026-02-17T06:42:47.294 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:47.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836650 2026-02-17T06:42:47.295 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836650 2026-02-17T06:42:47.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836650' 2026-02-17T06:42:47.296 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:47.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836648 -lt 21474836650 2026-02-17T06:42:47.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:48.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:48.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:48.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836651 -lt 21474836650 2026-02-17T06:42:48.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:48.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705826 2026-02-17T06:42:48.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:48.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:48.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705826 2026-02-17T06:42:48.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:48.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705826 2026-02-17T06:42:48.613 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705826 2026-02-17T06:42:48.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705826' 2026-02-17T06:42:48.613 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:48.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705826 -lt 38654705826 2026-02-17T06:42:48.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:48.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542298 2026-02-17T06:42:48.777 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:48.778 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:48.778 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542298 2026-02-17T06:42:48.779 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:48.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542298 2026-02-17T06:42:48.780 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542298 2026-02-17T06:42:48.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542298' 2026-02-17T06:42:48.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:48.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542299 -lt 60129542298 2026-02-17T06:42:48.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:48.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:48.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:49.082 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:49.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:49.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:49.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:49.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:49.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:49.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:49.244 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:49.244 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:49.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:49.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:49.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:49.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836653 2026-02-17T06:42:49.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836653 2026-02-17T06:42:49.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836653' 2026-02-17T06:42:49.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:49.316 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:49.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705829 2026-02-17T06:42:49.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705829 2026-02-17T06:42:49.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836653 1-38654705829' 2026-02-17T06:42:49.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:49.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:49.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542301 2026-02-17T06:42:49.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542301 2026-02-17T06:42:49.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836653 1-38654705829 2-60129542301' 2026-02-17T06:42:49.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:49.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836653 2026-02-17T06:42:49.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:49.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:49.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836653 2026-02-17T06:42:49.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:49.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836653 2026-02-17T06:42:49.463 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836653 2026-02-17T06:42:49.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836653' 2026-02-17T06:42:49.464 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:49.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836651 -lt 21474836653 2026-02-17T06:42:49.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:50.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:50.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:50.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836654 -lt 21474836653 2026-02-17T06:42:50.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:50.770 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705829 2026-02-17T06:42:50.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:50.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:50.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705829 2026-02-17T06:42:50.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:50.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705829 2026-02-17T06:42:50.774 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705829 2026-02-17T06:42:50.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705829' 2026-02-17T06:42:50.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:50.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705829 -lt 38654705829 2026-02-17T06:42:50.926 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:50.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542301 2026-02-17T06:42:50.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:50.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:50.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542301 2026-02-17T06:42:50.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:50.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542301 2026-02-17T06:42:50.930 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542301 2026-02-17T06:42:50.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542301' 2026-02-17T06:42:50.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:51.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542302 -lt 60129542301 2026-02-17T06:42:51.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:51.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:51.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:51.225 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:51.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:51.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:51.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:51.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:51.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:51.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:51.388 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:51.388 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:51.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:51.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:51.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:51.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836656 2026-02-17T06:42:51.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836656 2026-02-17T06:42:51.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836656' 2026-02-17T06:42:51.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:51.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:51.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705832 2026-02-17T06:42:51.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705832 2026-02-17T06:42:51.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836656 1-38654705832' 2026-02-17T06:42:51.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:51.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:51.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542304 2026-02-17T06:42:51.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542304 2026-02-17T06:42:51.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836656 1-38654705832 2-60129542304' 2026-02-17T06:42:51.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:51.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836656 2026-02-17T06:42:51.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:51.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:51.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836656 2026-02-17T06:42:51.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:51.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836656 2026-02-17T06:42:51.607 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836656 2026-02-17T06:42:51.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836656' 2026-02-17T06:42:51.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:51.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836654 -lt 21474836656 2026-02-17T06:42:51.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:52.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:52.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:52.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836657 -lt 21474836656 2026-02-17T06:42:52.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:52.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705832 2026-02-17T06:42:52.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:52.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:52.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705832 2026-02-17T06:42:52.924 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:52.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705832 2026-02-17T06:42:52.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705832' 2026-02-17T06:42:52.926 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705832 2026-02-17T06:42:52.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:53.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705832 -lt 38654705832 2026-02-17T06:42:53.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:53.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542304 2026-02-17T06:42:53.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:53.092 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:53.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542304 2026-02-17T06:42:53.093 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:53.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542304 2026-02-17T06:42:53.095 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542304 2026-02-17T06:42:53.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542304' 2026-02-17T06:42:53.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:53.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542305 -lt 60129542304 2026-02-17T06:42:53.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:53.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:53.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:53.385 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:53.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:53.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:53.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:53.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:53.395 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:53.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:53.548 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:53.548 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:53.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:53.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:53.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:53.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836660 2026-02-17T06:42:53.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836660 2026-02-17T06:42:53.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836660' 2026-02-17T06:42:53.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:53.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:53.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705835 2026-02-17T06:42:53.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705835 2026-02-17T06:42:53.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836660 1-38654705835' 2026-02-17T06:42:53.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:53.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:53.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542307 2026-02-17T06:42:53.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542307 2026-02-17T06:42:53.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836660 1-38654705835 2-60129542307' 2026-02-17T06:42:53.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:53.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836660 2026-02-17T06:42:53.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:53.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:53.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836660 2026-02-17T06:42:53.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:53.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836660 2026-02-17T06:42:53.768 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836660 2026-02-17T06:42:53.769 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836660' 2026-02-17T06:42:53.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:53.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836660 -lt 21474836660 2026-02-17T06:42:53.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:53.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705835 2026-02-17T06:42:53.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:53.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:53.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705835 2026-02-17T06:42:53.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:53.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705835 2026-02-17T06:42:53.935 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705835 2026-02-17T06:42:53.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705835' 2026-02-17T06:42:53.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:54.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705835 -lt 38654705835 2026-02-17T06:42:54.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:54.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542307 2026-02-17T06:42:54.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:54.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:54.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542307 2026-02-17T06:42:54.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:54.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542307 2026-02-17T06:42:54.087 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542307 2026-02-17T06:42:54.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542307' 2026-02-17T06:42:54.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:54.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542308 -lt 60129542307 2026-02-17T06:42:54.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:54.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:54.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:54.380 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:54.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:54.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:54.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:54.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:54.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:54.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:54.543 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:54.543 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:54.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:54.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:54.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:54.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836662 2026-02-17T06:42:54.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836662 2026-02-17T06:42:54.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836662' 2026-02-17T06:42:54.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:54.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:54.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705837 2026-02-17T06:42:54.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705837 2026-02-17T06:42:54.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836662 1-38654705837' 2026-02-17T06:42:54.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:54.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:54.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542309 2026-02-17T06:42:54.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542309 2026-02-17T06:42:54.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836662 1-38654705837 2-60129542309' 2026-02-17T06:42:54.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:54.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836662 2026-02-17T06:42:54.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:54.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:54.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836662 2026-02-17T06:42:54.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:54.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836662 2026-02-17T06:42:54.766 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836662 2026-02-17T06:42:54.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836662' 2026-02-17T06:42:54.767 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:54.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836660 -lt 21474836662 2026-02-17T06:42:54.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:55.918 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:55.918 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:56.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836663 -lt 21474836662 2026-02-17T06:42:56.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:56.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705837 2026-02-17T06:42:56.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:56.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:56.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705837 2026-02-17T06:42:56.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:56.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705837 2026-02-17T06:42:56.076 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705837 2026-02-17T06:42:56.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705837' 2026-02-17T06:42:56.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:56.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705838 -lt 38654705837 2026-02-17T06:42:56.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:56.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542309 2026-02-17T06:42:56.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:56.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:56.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542309 2026-02-17T06:42:56.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:56.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542309 2026-02-17T06:42:56.243 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542309' 2026-02-17T06:42:56.244 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542309 2026-02-17T06:42:56.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:56.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542311 -lt 60129542309 2026-02-17T06:42:56.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:56.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:56.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:56.540 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:56.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:56.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:56.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:56.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:56.550 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:56.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:56.698 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:56.698 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:56.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:56.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:56.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:56.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836665 2026-02-17T06:42:56.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836665 2026-02-17T06:42:56.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836665' 2026-02-17T06:42:56.768 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:56.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:56.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705840 2026-02-17T06:42:56.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705840 2026-02-17T06:42:56.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836665 1-38654705840' 2026-02-17T06:42:56.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:56.839 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:56.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542313 2026-02-17T06:42:56.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542313 2026-02-17T06:42:56.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836665 1-38654705840 2-60129542313' 2026-02-17T06:42:56.911 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:56.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836665 2026-02-17T06:42:56.912 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:56.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:56.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836665 2026-02-17T06:42:56.914 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:56.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836665 2026-02-17T06:42:56.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836665' 2026-02-17T06:42:56.916 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836665 2026-02-17T06:42:56.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:57.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836663 -lt 21474836665 2026-02-17T06:42:57.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:42:58.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:42:58.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:58.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836666 -lt 21474836665 2026-02-17T06:42:58.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:58.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705840 2026-02-17T06:42:58.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:58.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:42:58.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705840 2026-02-17T06:42:58.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:58.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705840 2026-02-17T06:42:58.236 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705840 2026-02-17T06:42:58.236 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705840' 2026-02-17T06:42:58.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:42:58.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705841 -lt 38654705840 2026-02-17T06:42:58.393 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:58.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542313 2026-02-17T06:42:58.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:58.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:42:58.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542313 2026-02-17T06:42:58.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:58.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542313 2026-02-17T06:42:58.397 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542313' 2026-02-17T06:42:58.398 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542313 2026-02-17T06:42:58.398 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:42:58.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542314 -lt 60129542313 2026-02-17T06:42:58.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:42:58.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:42:58.557 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:42:58.695 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:42:58.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:42:58.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:42:58.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:42:58.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:42:58.705 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:42:58.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:42:58.848 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:42:58.848 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:42:58.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:42:58.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:58.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:42:58.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836668 2026-02-17T06:42:58.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836668 2026-02-17T06:42:58.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836668' 2026-02-17T06:42:58.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:58.919 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:42:58.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705843 2026-02-17T06:42:58.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705843 2026-02-17T06:42:58.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836668 1-38654705843' 2026-02-17T06:42:58.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:42:58.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:42:59.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542316 2026-02-17T06:42:59.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542316 2026-02-17T06:42:59.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836668 1-38654705843 2-60129542316' 2026-02-17T06:42:59.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:42:59.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836668 2026-02-17T06:42:59.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:42:59.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:42:59.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836668 2026-02-17T06:42:59.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:42:59.067 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836668 2026-02-17T06:42:59.067 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836668 2026-02-17T06:42:59.068 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836668' 2026-02-17T06:42:59.068 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:42:59.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836666 -lt 21474836668 2026-02-17T06:42:59.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:00.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:00.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:00.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836669 -lt 21474836668 2026-02-17T06:43:00.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705843 2026-02-17T06:43:00.381 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:00.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:00.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705843 2026-02-17T06:43:00.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:00.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705843 2026-02-17T06:43:00.385 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705843 2026-02-17T06:43:00.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705843' 2026-02-17T06:43:00.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:00.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705844 -lt 38654705843 2026-02-17T06:43:00.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:00.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542316 2026-02-17T06:43:00.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:00.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:00.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542316 2026-02-17T06:43:00.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:00.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542316 2026-02-17T06:43:00.547 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542316 2026-02-17T06:43:00.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542316' 2026-02-17T06:43:00.547 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:00.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542317 -lt 60129542316 2026-02-17T06:43:00.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:00.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:00.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:00.849 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:00.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:00.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:00.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:00.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:00.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:01.007 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:01.007 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:01.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:01.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:01.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836671 2026-02-17T06:43:01.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836671 2026-02-17T06:43:01.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836671' 2026-02-17T06:43:01.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:01.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:01.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705847 2026-02-17T06:43:01.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705847 2026-02-17T06:43:01.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836671 1-38654705847' 2026-02-17T06:43:01.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:01.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:01.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542319 2026-02-17T06:43:01.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542319 2026-02-17T06:43:01.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836671 1-38654705847 2-60129542319' 2026-02-17T06:43:01.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:01.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836671 2026-02-17T06:43:01.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:01.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:01.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836671 2026-02-17T06:43:01.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:01.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836671 2026-02-17T06:43:01.227 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836671 2026-02-17T06:43:01.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836671' 2026-02-17T06:43:01.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:01.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836669 -lt 21474836671 2026-02-17T06:43:01.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:02.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:02.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:02.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836672 -lt 21474836671 2026-02-17T06:43:02.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:02.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705847 2026-02-17T06:43:02.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:02.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:02.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705847 2026-02-17T06:43:02.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:02.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705847 2026-02-17T06:43:02.533 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705847 2026-02-17T06:43:02.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705847' 2026-02-17T06:43:02.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:02.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705847 -lt 38654705847 2026-02-17T06:43:02.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:02.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542319 2026-02-17T06:43:02.692 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:02.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:02.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542319 2026-02-17T06:43:02.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:02.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542319 2026-02-17T06:43:02.695 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542319 2026-02-17T06:43:02.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542319' 2026-02-17T06:43:02.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:02.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542320 -lt 60129542319 2026-02-17T06:43:02.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:02.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:02.852 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:02.993 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:03.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:03.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:03.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:03.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:03.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:03.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:03.147 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:03.147 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:03.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:03.147 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:03.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:03.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836674 2026-02-17T06:43:03.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836674 2026-02-17T06:43:03.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836674' 2026-02-17T06:43:03.221 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:03.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:03.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705850 2026-02-17T06:43:03.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705850 2026-02-17T06:43:03.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836674 1-38654705850' 2026-02-17T06:43:03.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:03.291 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542322 2026-02-17T06:43:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542322 2026-02-17T06:43:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836674 1-38654705850 2-60129542322' 2026-02-17T06:43:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:03.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836674 2026-02-17T06:43:03.363 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:03.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:03.364 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836674 2026-02-17T06:43:03.365 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:03.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836674 2026-02-17T06:43:03.366 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836674 2026-02-17T06:43:03.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836674' 2026-02-17T06:43:03.367 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:03.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836672 -lt 21474836674 2026-02-17T06:43:03.517 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:04.518 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:04.519 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:04.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836675 -lt 21474836674 2026-02-17T06:43:04.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:04.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705850 2026-02-17T06:43:04.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:04.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:04.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705850 2026-02-17T06:43:04.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:04.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705850 2026-02-17T06:43:04.676 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705850' 2026-02-17T06:43:04.676 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705850 2026-02-17T06:43:04.677 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:04.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705850 -lt 38654705850 2026-02-17T06:43:04.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:04.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542322 2026-02-17T06:43:04.838 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:04.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:04.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542322 2026-02-17T06:43:04.840 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:04.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542322 2026-02-17T06:43:04.841 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542322 2026-02-17T06:43:04.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542322' 2026-02-17T06:43:04.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:05.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542323 -lt 60129542322 2026-02-17T06:43:05.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:05.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:05.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:05.146 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:05.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:05.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:05.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:05.156 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:05.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:05.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:05.307 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:05.307 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:05.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:05.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:05.307 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:05.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836677 2026-02-17T06:43:05.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836677 2026-02-17T06:43:05.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836677' 2026-02-17T06:43:05.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:05.379 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:05.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705853 2026-02-17T06:43:05.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705853 2026-02-17T06:43:05.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836677 1-38654705853' 2026-02-17T06:43:05.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:05.453 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:05.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542325 2026-02-17T06:43:05.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542325 2026-02-17T06:43:05.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836677 1-38654705853 2-60129542325' 2026-02-17T06:43:05.525 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:05.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836677 2026-02-17T06:43:05.526 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:05.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:05.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836677 2026-02-17T06:43:05.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:05.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836677 2026-02-17T06:43:05.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836677' 2026-02-17T06:43:05.529 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836677 2026-02-17T06:43:05.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:05.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836675 -lt 21474836677 2026-02-17T06:43:05.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:06.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:06.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:06.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836678 -lt 21474836677 2026-02-17T06:43:06.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:06.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705853 2026-02-17T06:43:06.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:06.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:06.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705853 2026-02-17T06:43:06.851 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:06.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705853 2026-02-17T06:43:06.853 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705853 2026-02-17T06:43:06.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705853' 2026-02-17T06:43:06.853 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:07.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705853 -lt 38654705853 2026-02-17T06:43:07.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:07.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542325 2026-02-17T06:43:07.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:07.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:07.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542325 2026-02-17T06:43:07.019 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:07.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542325 2026-02-17T06:43:07.021 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542325 2026-02-17T06:43:07.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542325' 2026-02-17T06:43:07.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:07.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542326 -lt 60129542325 2026-02-17T06:43:07.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:07.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:07.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:07.317 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:07.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:07.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:07.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:07.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:07.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:07.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:07.477 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:07.477 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:07.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:07.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:07.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:07.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836680 2026-02-17T06:43:07.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836680 2026-02-17T06:43:07.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836680' 2026-02-17T06:43:07.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:07.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:07.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705856 2026-02-17T06:43:07.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705856 2026-02-17T06:43:07.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836680 1-38654705856' 2026-02-17T06:43:07.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:07.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:07.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542328 2026-02-17T06:43:07.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542328 2026-02-17T06:43:07.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836680 1-38654705856 2-60129542328' 2026-02-17T06:43:07.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:07.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836680 2026-02-17T06:43:07.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:07.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:07.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836680 2026-02-17T06:43:07.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:07.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836680 2026-02-17T06:43:07.690 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836680 2026-02-17T06:43:07.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836680' 2026-02-17T06:43:07.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:07.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836681 -lt 21474836680 2026-02-17T06:43:07.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:07.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705856 2026-02-17T06:43:07.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:07.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:07.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705856 2026-02-17T06:43:07.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:07.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705856 2026-02-17T06:43:07.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705856' 2026-02-17T06:43:07.849 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705856 2026-02-17T06:43:07.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:08.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705856 -lt 38654705856 2026-02-17T06:43:08.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:08.006 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542328 2026-02-17T06:43:08.007 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:08.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:08.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542328 2026-02-17T06:43:08.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:08.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542328 2026-02-17T06:43:08.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542328' 2026-02-17T06:43:08.010 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542328 2026-02-17T06:43:08.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:08.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542329 -lt 60129542328 2026-02-17T06:43:08.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:08.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:08.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:08.304 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:08.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:08.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:08.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:08.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:08.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:08.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:08.459 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:08.459 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:08.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:08.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:08.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:08.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836682 2026-02-17T06:43:08.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836682 2026-02-17T06:43:08.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836682' 2026-02-17T06:43:08.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:08.530 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:08.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705858 2026-02-17T06:43:08.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705858 2026-02-17T06:43:08.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836682 1-38654705858' 2026-02-17T06:43:08.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:08.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:08.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542330 2026-02-17T06:43:08.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542330 2026-02-17T06:43:08.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836682 1-38654705858 2-60129542330' 2026-02-17T06:43:08.671 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:08.671 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836682 2026-02-17T06:43:08.672 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:08.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:08.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836682 2026-02-17T06:43:08.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:08.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836682 2026-02-17T06:43:08.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836682' 2026-02-17T06:43:08.675 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836682 2026-02-17T06:43:08.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:08.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836681 -lt 21474836682 2026-02-17T06:43:08.825 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:09.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:09.826 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:09.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836684 -lt 21474836682 2026-02-17T06:43:09.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:09.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705858 2026-02-17T06:43:09.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:09.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:09.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705858 2026-02-17T06:43:09.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:09.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705858 2026-02-17T06:43:09.983 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705858 2026-02-17T06:43:09.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705858' 2026-02-17T06:43:09.984 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:10.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705859 -lt 38654705858 2026-02-17T06:43:10.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:10.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542330 2026-02-17T06:43:10.145 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:10.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:10.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542330 2026-02-17T06:43:10.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:10.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542330 2026-02-17T06:43:10.148 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542330 2026-02-17T06:43:10.148 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542330' 2026-02-17T06:43:10.148 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542332 -lt 60129542330 2026-02-17T06:43:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:10.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:10.449 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:10.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:10.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:10.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:10.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:10.458 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:10.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:10.608 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:10.608 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:10.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:10.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:10.608 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:10.678 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836686 2026-02-17T06:43:10.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836686 2026-02-17T06:43:10.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686' 2026-02-17T06:43:10.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:10.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:10.753 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705861 2026-02-17T06:43:10.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705861 2026-02-17T06:43:10.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686 1-38654705861' 2026-02-17T06:43:10.754 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:10.754 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:10.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542334 2026-02-17T06:43:10.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542334 2026-02-17T06:43:10.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686 1-38654705861 2-60129542334' 2026-02-17T06:43:10.823 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:10.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836686 2026-02-17T06:43:10.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:10.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:10.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836686 2026-02-17T06:43:10.825 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:10.826 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836686 2026-02-17T06:43:10.827 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836686 2026-02-17T06:43:10.828 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836686' 2026-02-17T06:43:10.828 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:10.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836684 -lt 21474836686 2026-02-17T06:43:10.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:11.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:11.977 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:12.130 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836687 -lt 21474836686 2026-02-17T06:43:12.131 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:12.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705861 2026-02-17T06:43:12.131 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:12.132 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:12.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705861 2026-02-17T06:43:12.133 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:12.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705861 2026-02-17T06:43:12.135 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705861 2026-02-17T06:43:12.135 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705861' 2026-02-17T06:43:12.135 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:12.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705862 -lt 38654705861 2026-02-17T06:43:12.284 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:12.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542334 2026-02-17T06:43:12.285 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:12.286 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:12.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542334 2026-02-17T06:43:12.287 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:12.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542334 2026-02-17T06:43:12.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542334' 2026-02-17T06:43:12.289 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542334 2026-02-17T06:43:12.289 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:12.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542335 -lt 60129542334 2026-02-17T06:43:12.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:12.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:12.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:12.587 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:12.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:12.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:12.596 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:12.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:12.597 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:12.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:12.748 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:12.748 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:12.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:12.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:12.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:12.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836689 2026-02-17T06:43:12.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836689 2026-02-17T06:43:12.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836689' 2026-02-17T06:43:12.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:12.818 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:12.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705864 2026-02-17T06:43:12.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705864 2026-02-17T06:43:12.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836689 1-38654705864' 2026-02-17T06:43:12.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:12.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:12.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542337 2026-02-17T06:43:12.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542337 2026-02-17T06:43:12.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836689 1-38654705864 2-60129542337' 2026-02-17T06:43:12.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:12.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836689 2026-02-17T06:43:12.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:12.965 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:12.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836689 2026-02-17T06:43:12.966 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:12.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836689 2026-02-17T06:43:12.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836689' 2026-02-17T06:43:12.968 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836689 2026-02-17T06:43:12.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:13.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836687 -lt 21474836689 2026-02-17T06:43:13.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:14.127 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:14.128 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:14.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836690 -lt 21474836689 2026-02-17T06:43:14.277 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:14.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705864 2026-02-17T06:43:14.278 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:14.279 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:14.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705864 2026-02-17T06:43:14.280 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:14.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705864 2026-02-17T06:43:14.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705864' 2026-02-17T06:43:14.282 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705864 2026-02-17T06:43:14.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:14.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705865 -lt 38654705864 2026-02-17T06:43:14.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:14.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542337 2026-02-17T06:43:14.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:14.434 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:14.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542337 2026-02-17T06:43:14.435 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:14.436 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542337 2026-02-17T06:43:14.436 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542337 2026-02-17T06:43:14.437 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542337' 2026-02-17T06:43:14.437 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:14.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542338 -lt 60129542337 2026-02-17T06:43:14.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:14.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:14.593 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:14.734 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:14.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:14.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:14.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:14.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:14.743 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:14.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:14.894 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:14.894 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:14.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:14.894 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:14.894 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:14.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836692 2026-02-17T06:43:14.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836692 2026-02-17T06:43:14.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836692' 2026-02-17T06:43:14.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:14.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:15.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705867 2026-02-17T06:43:15.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705867 2026-02-17T06:43:15.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836692 1-38654705867' 2026-02-17T06:43:15.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:15.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:15.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542340 2026-02-17T06:43:15.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542340 2026-02-17T06:43:15.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836692 1-38654705867 2-60129542340' 2026-02-17T06:43:15.106 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:15.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836692 2026-02-17T06:43:15.107 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:15.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:15.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836692 2026-02-17T06:43:15.108 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:15.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836692 2026-02-17T06:43:15.110 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836692 2026-02-17T06:43:15.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836692' 2026-02-17T06:43:15.110 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:15.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836690 -lt 21474836692 2026-02-17T06:43:15.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:16.271 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:16.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:16.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836693 -lt 21474836692 2026-02-17T06:43:16.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:16.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705867 2026-02-17T06:43:16.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:16.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:16.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705867 2026-02-17T06:43:16.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:16.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705867 2026-02-17T06:43:16.425 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705867 2026-02-17T06:43:16.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705867' 2026-02-17T06:43:16.425 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:16.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705868 -lt 38654705867 2026-02-17T06:43:16.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:16.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542340 2026-02-17T06:43:16.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:16.592 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:16.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542340 2026-02-17T06:43:16.592 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:16.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542340 2026-02-17T06:43:16.594 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542340 2026-02-17T06:43:16.594 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542340' 2026-02-17T06:43:16.595 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:16.742 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542341 -lt 60129542340 2026-02-17T06:43:16.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:16.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:16.743 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:16.887 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:16.897 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:16.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:16.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:16.898 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:16.898 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:17.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:17.045 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:17.045 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:17.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:17.045 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:17.046 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:17.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836695 2026-02-17T06:43:17.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836695 2026-02-17T06:43:17.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836695' 2026-02-17T06:43:17.116 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:17.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:17.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705871 2026-02-17T06:43:17.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705871 2026-02-17T06:43:17.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836695 1-38654705871' 2026-02-17T06:43:17.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:17.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:17.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542343 2026-02-17T06:43:17.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542343 2026-02-17T06:43:17.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836695 1-38654705871 2-60129542343' 2026-02-17T06:43:17.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:17.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836695 2026-02-17T06:43:17.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:17.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:17.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836695 2026-02-17T06:43:17.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:17.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836695 2026-02-17T06:43:17.262 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836695 2026-02-17T06:43:17.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836695' 2026-02-17T06:43:17.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:17.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836693 -lt 21474836695 2026-02-17T06:43:17.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:18.413 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:18.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:18.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836696 -lt 21474836695 2026-02-17T06:43:18.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:18.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705871 2026-02-17T06:43:18.565 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:18.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:18.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705871 2026-02-17T06:43:18.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:18.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705871 2026-02-17T06:43:18.568 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705871 2026-02-17T06:43:18.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705871' 2026-02-17T06:43:18.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:18.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705871 -lt 38654705871 2026-02-17T06:43:18.727 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:18.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542343 2026-02-17T06:43:18.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:18.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:18.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542343 2026-02-17T06:43:18.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:18.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542343 2026-02-17T06:43:18.731 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542343 2026-02-17T06:43:18.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542343' 2026-02-17T06:43:18.731 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:18.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542344 -lt 60129542343 2026-02-17T06:43:18.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:18.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:18.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:19.028 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:19.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:19.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:19.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:19.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:19.037 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:19.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:19.187 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:19.187 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:19.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:19.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:19.187 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:19.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836698 2026-02-17T06:43:19.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836698 2026-02-17T06:43:19.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836698' 2026-02-17T06:43:19.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:19.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:19.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705874 2026-02-17T06:43:19.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705874 2026-02-17T06:43:19.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836698 1-38654705874' 2026-02-17T06:43:19.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:19.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:19.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542346 2026-02-17T06:43:19.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542346 2026-02-17T06:43:19.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836698 1-38654705874 2-60129542346' 2026-02-17T06:43:19.400 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:19.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836698 2026-02-17T06:43:19.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:19.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:19.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836698 2026-02-17T06:43:19.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:19.404 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836698 2026-02-17T06:43:19.404 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836698 2026-02-17T06:43:19.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836698' 2026-02-17T06:43:19.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:19.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836696 -lt 21474836698 2026-02-17T06:43:19.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:20.559 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:20.559 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:20.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836699 -lt 21474836698 2026-02-17T06:43:20.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:20.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705874 2026-02-17T06:43:20.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:20.713 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:20.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705874 2026-02-17T06:43:20.713 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:20.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705874 2026-02-17T06:43:20.715 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705874 2026-02-17T06:43:20.715 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705874' 2026-02-17T06:43:20.716 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:20.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705874 -lt 38654705874 2026-02-17T06:43:20.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:20.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542346 2026-02-17T06:43:20.865 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:20.866 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:20.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542346 2026-02-17T06:43:20.867 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:20.868 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542346 2026-02-17T06:43:20.868 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542346 2026-02-17T06:43:20.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542346' 2026-02-17T06:43:20.869 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:21.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542347 -lt 60129542346 2026-02-17T06:43:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:21.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:21.169 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:21.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:21.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:21.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:21.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:21.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:21.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:21.326 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:21.326 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:21.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:21.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:21.326 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:21.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836701 2026-02-17T06:43:21.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836701 2026-02-17T06:43:21.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836701' 2026-02-17T06:43:21.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:21.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:21.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705877 2026-02-17T06:43:21.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705877 2026-02-17T06:43:21.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836701 1-38654705877' 2026-02-17T06:43:21.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:21.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:21.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542349 2026-02-17T06:43:21.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542349 2026-02-17T06:43:21.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836701 1-38654705877 2-60129542349' 2026-02-17T06:43:21.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:21.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836701 2026-02-17T06:43:21.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:21.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:21.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836701 2026-02-17T06:43:21.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:21.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836701 2026-02-17T06:43:21.545 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836701 2026-02-17T06:43:21.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836701' 2026-02-17T06:43:21.545 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:21.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836699 -lt 21474836701 2026-02-17T06:43:21.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:22.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:22.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:22.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836702 -lt 21474836701 2026-02-17T06:43:22.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:22.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705877 2026-02-17T06:43:22.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:22.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:22.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705877 2026-02-17T06:43:22.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:22.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705877 2026-02-17T06:43:22.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705877' 2026-02-17T06:43:22.862 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705877 2026-02-17T06:43:22.862 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:23.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705877 -lt 38654705877 2026-02-17T06:43:23.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:23.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542349 2026-02-17T06:43:23.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:23.018 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:23.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542349 2026-02-17T06:43:23.018 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:23.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542349 2026-02-17T06:43:23.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542349' 2026-02-17T06:43:23.020 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542349 2026-02-17T06:43:23.020 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:23.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542350 -lt 60129542349 2026-02-17T06:43:23.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:23.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:23.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:23.322 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:23.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:23.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:23.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:23.332 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:23.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:23.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:23.482 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:23.482 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:23.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:23.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:23.483 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:23.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836704 2026-02-17T06:43:23.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836704 2026-02-17T06:43:23.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836704' 2026-02-17T06:43:23.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:23.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:23.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705880 2026-02-17T06:43:23.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705880 2026-02-17T06:43:23.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836704 1-38654705880' 2026-02-17T06:43:23.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:23.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:23.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542352 2026-02-17T06:43:23.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542352 2026-02-17T06:43:23.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836704 1-38654705880 2-60129542352' 2026-02-17T06:43:23.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:23.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836704 2026-02-17T06:43:23.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:23.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:23.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836704 2026-02-17T06:43:23.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:23.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836704 2026-02-17T06:43:23.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836704' 2026-02-17T06:43:23.699 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836704 2026-02-17T06:43:23.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:23.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836705 -lt 21474836704 2026-02-17T06:43:23.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:23.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705880 2026-02-17T06:43:23.854 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:23.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:23.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705880 2026-02-17T06:43:23.856 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:23.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705880 2026-02-17T06:43:23.858 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705880 2026-02-17T06:43:23.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705880' 2026-02-17T06:43:23.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:24.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705880 -lt 38654705880 2026-02-17T06:43:24.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:24.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542352 2026-02-17T06:43:24.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:24.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:24.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542352 2026-02-17T06:43:24.011 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:24.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542352 2026-02-17T06:43:24.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542352' 2026-02-17T06:43:24.013 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542352 2026-02-17T06:43:24.013 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:24.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542353 -lt 60129542352 2026-02-17T06:43:24.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:24.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:24.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:24.313 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:24.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T06:43:24.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T06:43:24.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T06:43:24.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:43:24.322 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:43:24.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:43:24.471 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:43:24.471 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:43:24.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:43:24.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:24.472 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:43:24.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836706 2026-02-17T06:43:24.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836706 2026-02-17T06:43:24.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836706' 2026-02-17T06:43:24.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:24.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:43:24.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705882 2026-02-17T06:43:24.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705882 2026-02-17T06:43:24.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836706 1-38654705882' 2026-02-17T06:43:24.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:43:24.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:43:24.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542354 2026-02-17T06:43:24.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542354 2026-02-17T06:43:24.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836706 1-38654705882 2-60129542354' 2026-02-17T06:43:24.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:24.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836706 2026-02-17T06:43:24.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:24.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:43:24.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836706 2026-02-17T06:43:24.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:24.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836706 2026-02-17T06:43:24.688 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836706 2026-02-17T06:43:24.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836706' 2026-02-17T06:43:24.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:24.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836705 -lt 21474836706 2026-02-17T06:43:24.847 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:43:25.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:43:25.849 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:43:25.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836708 -lt 21474836706 2026-02-17T06:43:25.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:26.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705882 2026-02-17T06:43:26.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:26.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:43:26.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705882 2026-02-17T06:43:26.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:26.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705882 2026-02-17T06:43:26.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705882' 2026-02-17T06:43:26.003 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705882 2026-02-17T06:43:26.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:43:26.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705883 -lt 38654705882 2026-02-17T06:43:26.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:43:26.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542354 2026-02-17T06:43:26.166 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:43:26.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:43:26.167 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542354 2026-02-17T06:43:26.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:43:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542354 2026-02-17T06:43:26.169 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542354 2026-02-17T06:43:26.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542354' 2026-02-17T06:43:26.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:43:26.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542356 -lt 60129542354 2026-02-17T06:43:26.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T06:43:26.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T06:43:26.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T06:43:26.471 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:43:26.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:383: _scrub_abort: break 2026-02-17T06:43:26.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:385: _scrub_abort: set +o pipefail 2026-02-17T06:43:26.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:387: _scrub_abort: sleep 5 2026-02-17T06:43:31.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:389: _scrub_abort: grep 'noscrub set, aborting' td/osd-scrub-test/osd.1.log 2026-02-17T06:43:31.490 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:41:29.095+0000 7fd63b69d640 10 osd.1 pg_epoch: 19 pg[1.0( v 18'1000 (0'0,18'1000] local-lis/les=15/17 n=1000 ec=15/15 lis/c=15/15 les/c/f=17/17/0 sis=15) [1,0,2] r=0 lpr=15 crt=18'1000 lcod 18'999 mlcod 18'999 active+clean+scrubbing [ 1.0: ] ] scrubber: noscrub set, aborting 2026-02-17T06:43:31.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: get_last_scrub_stamp 1.0 2026-02-17T06:43:31.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:31.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:31.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:31.491 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:31.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: local last_scrub=2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:31.638 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:396: _scrub_abort: ceph config set osd osd_scrub_sleep 0.1 2026-02-17T06:43:31.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:398: _scrub_abort: ceph osd unset noscrub 2026-02-17T06:43:31.982 INFO:tasks.workunit.client.0.trial074.stderr:noscrub is unset 2026-02-17T06:43:31.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:399: _scrub_abort: '[' scrub = deep-scrub ']' 2026-02-17T06:43:31.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:403: _scrub_abort: TIMEOUT=500 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:404: _scrub_abort: wait_for_scrub 1.0 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:31.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:32.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:32.143 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:33.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:33.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:33.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:33.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:33.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:33.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:33.146 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:33.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:33.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:34.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:34.298 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:34.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:34.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:34.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:34.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:34.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:34.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:34.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:35.447 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:35.448 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:35.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:35.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:35.448 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:35.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:35.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:35.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:35.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:36.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:36.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:36.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:36.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:36.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:36.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:36.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:36.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:36.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:37.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:37.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:37.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:37.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:37.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:37.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:37.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:37.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:37.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:38.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:38.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:38.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:38.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:38.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:38.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:38.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:39.048 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:39.049 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:40.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:40.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:40.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:40.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:40.050 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:40.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:40.051 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:40.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:40.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:41.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:41.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:41.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:41.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:41.197 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:41.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:41.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:41.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:41.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:42.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:42.346 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:42.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:42.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:42.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:42.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:42.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:42.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:42.493 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:43.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:43.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:43.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:43.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:43.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:43.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:43.495 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:43.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:43.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:44.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:44.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:44.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:44.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:44.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:44.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:44.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:44.790 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:44.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:45.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:45.792 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:45.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:45.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:45.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:45.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:45.793 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:45.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:45.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:46.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:46.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:46.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:46.942 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:46.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:46.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:46.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:47.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:47.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:48.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:48.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:48.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:48.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:48.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:48.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:48.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:48.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:48.239 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:49.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:49.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:49.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:49.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:49.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:49.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:49.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:49.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:49.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:50.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:50.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:50.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:50.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:50.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:50.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:50.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:50.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:50.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:51.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:51.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:51.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:51.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:51.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:51.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:51.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:51.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:51.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:52.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:52.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:52.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:52.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:52.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:52.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:52.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:52.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:52.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:53.845 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:53.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:53.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:53.846 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:53.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:53.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:53.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:53.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:53.998 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:54.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:54.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:55.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:55.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:55.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:55.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:55.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:55.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:55.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:56.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:56.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:56.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:56.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:56.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:56.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:56.155 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:56.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:56.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:57.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:57.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:57.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:57.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:57.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:57.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:57.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:57.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:57.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:58.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:58.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:58.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:58.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:58.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:58.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:58.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:58.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:58.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:43:59.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:43:59.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:43:59.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:43:59.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:43:59.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:43:59.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:43:59.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:43:59.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:43:59.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:00.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:00.751 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:00.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:00.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:00.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:00.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:00.752 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:00.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:00.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:01.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:01.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:01.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:01.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:01.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:01.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:01.902 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:02.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:02.050 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:03.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:03.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:03.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:03.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:03.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:03.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:03.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:03.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:03.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:04.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:04.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:04.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:04.201 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:04.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:04.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:04.202 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:04.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:04.347 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:05.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:05.348 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:05.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:05.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:05.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:05.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:05.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:05.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:05.495 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:06.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:06.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:06.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:06.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:06.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:06.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:06.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:06.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:06.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:07.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:07.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:07.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:07.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:07.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:07.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:07.646 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:07.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:07.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:08.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:08.796 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:08.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:08.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:08.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:08.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:08.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:08.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:08.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:09.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:09.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:09.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:09.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:09.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:09.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:09.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:10.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:10.095 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:11.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:11.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:11.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:11.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:11.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:11.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:11.098 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:11.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:11.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:12.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:12.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:12.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:12.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:12.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:12.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:12.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:12.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:12.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:13.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:13.392 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:13.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:13.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:13.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:13.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:13.393 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:13.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:13.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:14.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:14.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:14.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:14.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:14.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:14.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:14.540 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:14.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:14.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:15.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:15.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:15.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:15.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:15.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:15.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:15.689 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:15.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:15.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:16.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:16.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:16.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:16.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:16.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:16.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:16.841 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:16.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:16.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:17.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:17.996 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:17.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:17.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:17.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:17.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:17.997 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:18.149 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:18.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:19.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:19.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:19.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:19.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:19.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:19.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:19.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:19.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:19.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:20.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:20.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:20.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:20.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:20.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:20.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:20.304 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:20.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:20.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:21.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:21.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:21.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:21.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:21.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:21.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:21.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:21.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:21.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:22.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:22.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:22.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:22.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:22.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:22.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:22.620 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:22.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:22.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:23.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:23.771 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:23.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:23.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:23.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:23.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:23.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:23.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:23.923 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:24.924 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:24.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:24.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:24.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:24.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:24.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:24.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:25.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:25.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:26.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:26.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:26.077 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:26.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:26.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:26.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:26.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:26.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:26.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:27.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:27.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:27.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:27.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:27.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:27.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:27.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:27.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:27.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:28.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:28.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:28.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:28.382 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:28.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:28.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:28.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:28.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:28.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:29.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:29.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:29.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:29.534 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:29.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:29.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:29.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:29.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:29.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:30.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:30.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:30.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:30.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:30.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:30.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:30.688 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:30.839 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:30.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:31.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:31.841 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:31.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:31.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:31.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:31.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:31.842 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:31.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:32.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:32.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:32.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:32.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:32.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:32.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:32.996 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:33.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T06:39:43.331339+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:33.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:44:34.145 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:44:34.146 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:44:34.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:44:34.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:44:34.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:44:34.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:44:34.147 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:44:34.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:44:32.146730+0000 '>' 2026-02-10T06:39:43.331339+0000 2026-02-17T06:44:34.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:44:34.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:405: _scrub_abort: perf_counters td/osd-scrub-test 3 2026-02-17T06:44:34.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:44:34.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T06:44:34.297 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T06:44:34.298 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T06:44:34.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:44:34.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:44:34.299 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:44:34.363 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:44:34.363 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:44:34.363 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.363 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.363 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:44:34.363 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.364 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.365 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.366 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.367 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.368 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.369 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.370 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.371 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.372 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.373 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:44:34.374 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.375 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.376 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.377 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.377 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.377 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.377 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.377 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.377 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:44:34.378 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:44:34.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:44:34.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:44:34.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:44:34.438 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:44:34.438 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:44:34.438 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.438 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.438 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:44:34.438 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:44:34.438 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.439 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.440 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.441 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.442 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.443 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.444 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.445 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.446 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.447 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.448 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 2, 2026-02-17T06:44:34.449 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 2, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 1, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 55.079301803, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 55.079301803 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 1, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 4.99994116, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 4.99994116 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.450 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.451 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 11, 2026-02-17T06:44:34.451 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.451 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.451 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.451 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 2, 2026-02-17T06:44:34.451 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 2, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.452 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.453 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.453 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:44:34.453 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.453 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.453 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:44:34.453 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:44:34.453 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:44:34.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T06:44:34.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:44:34.509 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.510 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.511 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.512 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:44:34.513 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.514 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.515 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.516 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.517 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.518 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.519 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.520 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:44:34.521 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:44:34.522 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:44:34.523 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:44:34.523 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:44:34.523 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:44:34.523 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:44:34.523 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:44:34.524 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:44:34.525 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.525 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:44:34.525 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:44:34.525 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:44:34.525 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_scrub_abort ------------------ 2026-02-17T06:44:34.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_abort ------------------' 2026-02-17T06:44:34.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:44:34.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:44:34.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:44:34.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:44:34.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:44:34.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:44:34.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:44:34.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:44:34.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:44:34.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:44:34.629 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:44:34.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:44:34.630 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:44:34.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:44:34.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:44:34.631 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:44:34.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:44:34.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:44:34.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:44:34.633 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:44:34.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:44:34.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:44:34.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:44:34.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:44:34.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:34.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:34.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:44:34.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:44:34.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:44:34.689 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_scrub_abort ------------------ 2026-02-17T06:44:34.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_abort ------------------' 2026-02-17T06:44:34.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:44:34.689 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_scrub_extended_sleep ------------------- 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_extended_sleep -------------------' 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:44:34.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:44:34.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:44:34.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:44:34.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:44:34.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:44:34.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:44:34.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:44:34.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:44:34.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:44:34.693 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:44:34.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:44:34.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:44:34.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:44:34.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:44:34.695 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:44:34.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:44:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:44:34.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:44:34.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:44:34.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:34.700 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:34.700 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:44:34.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:44:34.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:44:34.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:44:34.702 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:44:34.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:34.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:34.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:44:34.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:44:34.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:44:34.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:44:34.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:44:34.706 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_scrub_extended_sleep ----------------------- 2026-02-17T06:44:34.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_extended_sleep -----------------------' 2026-02-17T06:44:34.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_extended_sleep td/osd-scrub-test 2026-02-17T06:44:34.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:209: TEST_scrub_extended_sleep: local dir=td/osd-scrub-test 2026-02-17T06:44:34.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:210: TEST_scrub_extended_sleep: local poolname=test 2026-02-17T06:44:34.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:211: TEST_scrub_extended_sleep: local OSDS=3 2026-02-17T06:44:34.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:212: TEST_scrub_extended_sleep: local objects=15 2026-02-17T06:44:34.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:214: TEST_scrub_extended_sleep: TESTDATA=testdata.507902 2026-02-17T06:44:34.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:216: TEST_scrub_extended_sleep: date +%w 2026-02-17T06:44:34.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:216: TEST_scrub_extended_sleep: DAY=2 2026-02-17T06:44:34.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:218: TEST_scrub_extended_sleep: '[' 2 -ge 4 ']' 2026-02-17T06:44:34.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:223: TEST_scrub_extended_sleep: expr 2 + 2 2026-02-17T06:44:34.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:223: TEST_scrub_extended_sleep: DAY_START=4 2026-02-17T06:44:34.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:224: TEST_scrub_extended_sleep: expr 2 + 3 2026-02-17T06:44:34.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:224: TEST_scrub_extended_sleep: DAY_END=5 2026-02-17T06:44:34.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:226: TEST_scrub_extended_sleep: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T06:44:34.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:44:34.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:44:34.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:44:34.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:44:34.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:44:34.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T06:44:34.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:44:34.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:44:34.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:44:34.732 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:44:34.732 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:34.733 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:34.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:34.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T06:44:34.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:44:34.764 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:44:34.765 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:44:34.766 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:34.766 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:34.766 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:44:34.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:44:34.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:44:34.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:44:34.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:44:34.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:44:34.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:44:34.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:44:34.823 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:44:34.823 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:44:34.823 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:44:34.823 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:44:34.824 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:34.824 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:34.824 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:44:34.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:44:34.824 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:44:34.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:227: TEST_scrub_extended_sleep: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:44:34.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:44:34.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:44:34.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:44:34.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:44:34.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:44:34.879 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:44:34.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:44:34.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:44:34.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:44:34.973 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:44:34.973 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:34.973 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:34.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:34.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:44:34.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:44:34.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:229: TEST_scrub_extended_sleep: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T06:44:34.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:230: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 ' 2026-02-17T06:44:34.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:231: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 ' 2026-02-17T06:44:34.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:232: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 ' 2026-02-17T06:44:34.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:233: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_op_queue=wpq --osd_scrub_end_week_day=5 ' 2026-02-17T06:44:34.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:234: TEST_scrub_extended_sleep: ceph_osd_args+=--bluestore_cache_autotune=false 2026-02-17T06:44:34.995 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: expr 3 - 1 2026-02-17T06:44:34.995 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: seq 0 2 2026-02-17T06:44:34.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:44:34.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:238: TEST_scrub_extended_sleep: run_osd td/osd-scrub-test 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:44:35.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:35.001 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false' 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:44:35.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:44:35.003 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=59e463b2-38a5-475e-abde-42ffc506c343 2026-02-17T06:44:35.003 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 59e463b2-38a5-475e-abde-42ffc506c343 2026-02-17T06:44:35.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 59e463b2-38a5-475e-abde-42ffc506c343' 2026-02-17T06:44:35.004 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:44:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBTDpRpaQJTABAAw+9pbdqD3D2bScJh3afE8Q== 2026-02-17T06:44:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBTDpRpaQJTABAAw+9pbdqD3D2bScJh3afE8Q=="}' 2026-02-17T06:44:35.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 59e463b2-38a5-475e-abde-42ffc506c343 -i td/osd-scrub-test/0/new.json 2026-02-17T06:44:35.099 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:44:35.107 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:44:35.108 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false --mkfs --key AQBTDpRpaQJTABAAw+9pbdqD3D2bScJh3afE8Q== --osd-uuid 59e463b2-38a5-475e-abde-42ffc506c343 2026-02-17T06:44:35.125 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:35.116+0000 7fe3b561c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:35.127 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:35.120+0000 7fe3b561c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:35.128 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:35.120+0000 7fe3b561c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:35.128 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:35.120+0000 7fe3b561c8c0 -1 bdev(0x564808f17c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:44:35.128 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:35.120+0000 7fe3b561c8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:44:37.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:44:37.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:44:37.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:44:37.959 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:44:37.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:44:38.153 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:44:38.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:44:38.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T06:44:38.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:44:38.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:44:38.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:44:38.180 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:38.172+0000 7f1e7716c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:38.181 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:38.172+0000 7f1e7716c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:38.183 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:38.176+0000 7f1e7716c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:38.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:44:38.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:44:38.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:44:38.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:44:38.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:44:38.303 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:38.303 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:44:38.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:44:38.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:38.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:44:38.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:39.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:39.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:39.456 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:44:39.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:44:39.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:39.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:44:39.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:39.672 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:39.664+0000 7f1e7716c8c0 -1 Falling back to public interface 2026-02-17T06:44:40.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:40.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:40.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:44:40.609 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:44:40.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:40.610 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:44:40.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:41.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:41.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:41.761 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:44:41.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:44:41.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:41.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:44:41.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:42.645 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:42.636+0000 7f1e7716c8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:44:42.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:42.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:44:42.914 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:44:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:42.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:44:43.065 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1582583959,v1:127.0.0.1:6803/1582583959] [v2:127.0.0.1:6804/1582583959,v1:127.0.0.1:6805/1582583959] exists,up 59e463b2-38a5-475e-abde-42ffc506c343 2026-02-17T06:44:43.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:44:43.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:44:43.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:44:43.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:44:43.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:238: TEST_scrub_extended_sleep: run_osd td/osd-scrub-test 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T06:44:43.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:44:43.069 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:44:43.070 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:44:43.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false' 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:44:43.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:44:43.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4a319348-78f7-47a8-9689-52990198cb0d 2026-02-17T06:44:43.073 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 4a319348-78f7-47a8-9689-52990198cb0d 2026-02-17T06:44:43.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 4a319348-78f7-47a8-9689-52990198cb0d' 2026-02-17T06:44:43.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:44:43.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBbDpRpfi54BBAAT0Kj27XOh3RvAluZirEJJA== 2026-02-17T06:44:43.081 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBbDpRpfi54BBAAT0Kj27XOh3RvAluZirEJJA=="}' 2026-02-17T06:44:43.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4a319348-78f7-47a8-9689-52990198cb0d -i td/osd-scrub-test/1/new.json 2026-02-17T06:44:43.236 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:44:43.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:44:43.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false --mkfs --key AQBbDpRpfi54BBAAT0Kj27XOh3RvAluZirEJJA== --osd-uuid 4a319348-78f7-47a8-9689-52990198cb0d 2026-02-17T06:44:43.265 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:43.256+0000 7f822b1e58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:43.266 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:43.256+0000 7f822b1e58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:43.267 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:43.260+0000 7f822b1e58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:43.268 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:43.260+0000 7f822b1e58c0 -1 bdev(0x56065b581c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:44:43.268 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:43.260+0000 7f822b1e58c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:44:45.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:44:45.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:44:45.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:44:45.564 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:44:45.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:44:45.757 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:44:45.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:44:45.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T06:44:45.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:44:45.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:44:45.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:44:45.784 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:45.776+0000 7fb4fe9748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:45.786 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:45.776+0000 7fb4fe9748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:45.787 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:45.780+0000 7fb4fe9748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:45.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:44:45.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:44:45.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:44:45.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:44:45.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:44:45.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:45.913 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:44:45.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:44:45.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:45.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:44:46.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:47.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:47.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:47.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:44:47.074 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:44:47.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:47.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:44:47.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:47.288 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:47.280+0000 7fb4fe9748c0 -1 Falling back to public interface 2026-02-17T06:44:48.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:48.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:48.230 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:44:48.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:44:48.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:48.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:44:48.382 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:49.383 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:44:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:44:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:49.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:44:49.536 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:50.275 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:50.268+0000 7fb4fe9748c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:44:50.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:50.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:50.538 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:44:50.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:44:50.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:50.538 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:44:50.690 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3583598016,v1:127.0.0.1:6811/3583598016] [v2:127.0.0.1:6812/3583598016,v1:127.0.0.1:6813/3583598016] exists,up 4a319348-78f7-47a8-9689-52990198cb0d 2026-02-17T06:44:50.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:238: TEST_scrub_extended_sleep: run_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:44:50.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:44:50.696 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:44:50.697 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:44:50.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:44:50.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false' 2026-02-17T06:44:50.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:44:50.698 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:44:50.698 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3b7f492e-5af0-41cf-b42f-c5f6c056eca2 2026-02-17T06:44:50.699 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 3b7f492e-5af0-41cf-b42f-c5f6c056eca2' 2026-02-17T06:44:50.699 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 3b7f492e-5af0-41cf-b42f-c5f6c056eca2 2026-02-17T06:44:50.699 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:44:50.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBiDpRptu7PKRAAYbfgpgQRG19T/7ms03qMwQ== 2026-02-17T06:44:50.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBiDpRptu7PKRAAYbfgpgQRG19T/7ms03qMwQ=="}' 2026-02-17T06:44:50.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3b7f492e-5af0-41cf-b42f-c5f6c056eca2 -i td/osd-scrub-test/2/new.json 2026-02-17T06:44:50.862 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:44:50.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T06:44:50.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false --mkfs --key AQBiDpRptu7PKRAAYbfgpgQRG19T/7ms03qMwQ== --osd-uuid 3b7f492e-5af0-41cf-b42f-c5f6c056eca2 2026-02-17T06:44:50.899 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:50.892+0000 7f93ceec08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:50.900 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:50.892+0000 7f93ceec08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:50.901 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:50.892+0000 7f93ceec08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:50.901 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:50.892+0000 7f93ceec08c0 -1 bdev(0x5614301fdc00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:44:50.901 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:50.892+0000 7f93ceec08c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:44:53.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T06:44:53.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:44:53.206 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:44:53.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:44:53.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:44:53.401 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:44:53.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:44:53.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T06:44:53.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:44:53.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:44:53.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:44:53.427 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:53.420+0000 7f48275208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:53.429 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:53.420+0000 7f48275208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:53.430 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:53.424+0000 7f48275208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:44:53.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:44:53.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:44:53.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:44:53.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:44:53.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:44:53.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:53.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:44:53.552 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:44:53.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:53.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:44:53.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:54.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:54.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:54.704 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:44:54.704 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:44:54.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:54.705 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:44:54.853 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:54.920 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:54.912+0000 7f48275208c0 -1 Falling back to public interface 2026-02-17T06:44:55.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:55.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:55.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:44:55.854 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:44:55.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:55.854 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:44:56.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:57.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:57.009 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:57.010 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:44:57.010 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:44:57.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:57.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:44:57.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:44:57.933 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:44:57.924+0000 7f48275208c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:44:58.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:44:58.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:44:58.160 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:44:58.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:44:58.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:44:58.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:44:58.309 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1103556294,v1:127.0.0.1:6819/1103556294] [v2:127.0.0.1:6820/1103556294,v1:127.0.0.1:6821/1103556294] exists,up 3b7f492e-5af0-41cf-b42f-c5f6c056eca2 2026-02-17T06:44:58.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:44:58.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:44:58.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:44:58.310 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:242: TEST_scrub_extended_sleep: create_pool test 1 1 2026-02-17T06:44:58.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:44:58.495 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:44:58.506 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:44:59.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:243: TEST_scrub_extended_sleep: wait_for_clean 2026-02-17T06:44:59.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:44:59.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:44:59.507 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:44:59.508 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:44:59.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:44:59.509 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:44:59.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:44:59.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:44:59.509 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:44:59.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:44:59.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:44:59.614 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:44:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:44:59.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:44:59.615 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:44:59.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:44:59.762 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:44:59.762 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:44:59.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:44:59.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:44:59.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:44:59.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T06:44:59.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T06:44:59.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T06:44:59.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:44:59.833 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:44:59.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672970 2026-02-17T06:44:59.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672970 2026-02-17T06:44:59.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-42949672970' 2026-02-17T06:44:59.905 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:44:59.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:44:59.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T06:44:59.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T06:44:59.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-42949672970 2-64424509442' 2026-02-17T06:44:59.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:44:59.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T06:44:59.976 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:44:59.977 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:44:59.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T06:44:59.978 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:44:59.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T06:44:59.980 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836498 2026-02-17T06:44:59.980 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T06:44:59.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:45:00.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836498 2026-02-17T06:45:00.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:45:01.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:45:01.142 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:45:01.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836498 2026-02-17T06:45:01.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:45:01.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672970 2026-02-17T06:45:01.292 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:45:01.294 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:45:01.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672970 2026-02-17T06:45:01.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:45:01.296 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672970 2026-02-17T06:45:01.297 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672970 2026-02-17T06:45:01.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-17T06:45:01.297 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:45:01.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672970 2026-02-17T06:45:01.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:45:01.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T06:45:01.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:45:01.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:45:01.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T06:45:01.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:45:01.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T06:45:01.462 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509442 2026-02-17T06:45:01.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T06:45:01.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:45:01.618 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509442 2026-02-17T06:45:01.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:45:01.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:45:01.619 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:45:01.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:45:01.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:45:01.795 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:45:01.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:45:01.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:45:01.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:45:01.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:45:01.796 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:45:01.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:45:01.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:45:01.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:45:01.944 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:246: TEST_scrub_extended_sleep: get_pg test SOMETHING 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test SOMETHING 2026-02-17T06:45:02.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:45:02.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:246: TEST_scrub_extended_sleep: local pgid=1.0 2026-02-17T06:45:02.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:247: TEST_scrub_extended_sleep: get_primary test SOMETHING 2026-02-17T06:45:02.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:45:02.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T06:45:02.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test SOMETHING 2026-02-17T06:45:02.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:45:02.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:247: TEST_scrub_extended_sleep: local primary=1 2026-02-17T06:45:02.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:248: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:02.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:02.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:02.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:02.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:02.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:248: TEST_scrub_extended_sleep: local last_scrub=2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:02.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:249: TEST_scrub_extended_sleep: ceph tell 1.0 schedule-scrub 2026-02-17T06:45:02.633 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:45:02.633 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:45:02.633 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:45:02.633 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-10T06:43:22.627468+0000" 2026-02-17T06:45:02.633 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:45:02.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:252: TEST_scrub_extended_sleep: PASSED=false 2026-02-17T06:45:02.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i=0 )) 2026-02-17T06:45:02.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i < 15 )) 2026-02-17T06:45:02.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:254: TEST_scrub_extended_sleep: grep -q 'scrub state.*, sleeping' td/osd-scrub-test/osd.1.log 2026-02-17T06:45:02.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:259: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:03.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:03.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i < 15 )) 2026-02-17T06:45:03.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:254: TEST_scrub_extended_sleep: grep -q 'scrub state.*, sleeping' td/osd-scrub-test/osd.1.log 2026-02-17T06:45:03.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:256: TEST_scrub_extended_sleep: PASSED=true 2026-02-17T06:45:03.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:257: TEST_scrub_extended_sleep: break 2026-02-17T06:45:03.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:263: TEST_scrub_extended_sleep: '[' true = false ']' 2026-02-17T06:45:03.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:269: TEST_scrub_extended_sleep: ceph tell osd.1 config set osd_scrub_begin_week_day 0 2026-02-17T06:45:03.717 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:45:03.717 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_begin_week_day = '' (not observed, change may require restart) " 2026-02-17T06:45:03.717 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:45:03.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:270: TEST_scrub_extended_sleep: ceph tell osd.1 config set osd_scrub_end_week_day 0 2026-02-17T06:45:03.791 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:45:03.792 INFO:tasks.workunit.client.0.trial074.stdout: "success": "osd_scrub_end_week_day = '' (not observed, change may require restart) " 2026-02-17T06:45:03.792 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:45:03.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:274: TEST_scrub_extended_sleep: count=0 2026-02-17T06:45:03.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:275: TEST_scrub_extended_sleep: PASSED=false 2026-02-17T06:45:03.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i=0 )) 2026-02-17T06:45:03.799 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:03.799 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 0 + 1 2026-02-17T06:45:03.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=1 2026-02-17T06:45:03.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:03.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:03.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:03.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:03.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:03.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-17T06:44:58.487100+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:03.955 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:04.956 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:04.957 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:04.957 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 1 + 1 2026-02-17T06:45:04.958 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=2 2026-02-17T06:45:04.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:04.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:04.959 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:04.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:04.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:05.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-17T06:44:58.487100+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:05.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:06.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:06.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:06.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 2 + 1 2026-02-17T06:45:06.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=3 2026-02-17T06:45:06.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:06.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:06.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:06.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:06.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:06.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:06.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:07.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:07.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:07.268 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 3 + 1 2026-02-17T06:45:07.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=4 2026-02-17T06:45:07.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:07.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:07.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:07.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:07.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:07.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:07.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:08.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:08.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:08.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 4 + 1 2026-02-17T06:45:08.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=5 2026-02-17T06:45:08.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:08.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:08.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:08.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:08.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:08.571 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:08.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:09.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:09.572 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:09.572 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 5 + 1 2026-02-17T06:45:09.573 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=6 2026-02-17T06:45:09.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:09.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:09.574 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:09.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:09.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:09.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:09.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:10.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:10.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:10.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 6 + 1 2026-02-17T06:45:10.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=7 2026-02-17T06:45:10.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:10.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:10.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:10.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:10.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:10.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:10.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:11.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:11.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:11.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 7 + 1 2026-02-17T06:45:11.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=8 2026-02-17T06:45:11.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:11.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:11.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:11.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:11.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:12.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:12.035 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:13.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:13.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:13.036 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 8 + 1 2026-02-17T06:45:13.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=9 2026-02-17T06:45:13.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:13.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:13.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:13.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:13.039 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:13.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:13.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:14.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:14.188 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:14.188 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 9 + 1 2026-02-17T06:45:14.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=10 2026-02-17T06:45:14.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:14.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:14.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:14.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:14.191 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:14.342 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:14.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:15.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:15.343 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:15.344 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 10 + 1 2026-02-17T06:45:15.344 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=11 2026-02-17T06:45:15.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:15.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:15.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:15.346 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:15.347 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:15.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:15.494 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:16.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:16.496 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:16.497 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 11 + 1 2026-02-17T06:45:16.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=12 2026-02-17T06:45:16.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:16.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:16.498 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:16.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:16.499 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:16.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:16.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:17.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:17.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:17.653 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 12 + 1 2026-02-17T06:45:17.654 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=13 2026-02-17T06:45:17.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:17.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:17.655 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:17.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:17.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:17.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:17.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:18.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:18.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:18.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 13 + 1 2026-02-17T06:45:18.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=14 2026-02-17T06:45:18.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:18.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:18.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:18.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:18.810 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:18.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:18.960 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:19.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:19.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:19.962 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 14 + 1 2026-02-17T06:45:19.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=15 2026-02-17T06:45:19.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:19.964 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:19.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:19.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:19.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:20.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:20.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:21.112 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:21.113 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:21.113 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 15 + 1 2026-02-17T06:45:21.114 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=16 2026-02-17T06:45:21.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:21.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:21.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:21.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:21.116 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:21.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:21.268 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:22.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:22.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:22.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 16 + 1 2026-02-17T06:45:22.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=17 2026-02-17T06:45:22.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:22.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:22.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:22.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:22.272 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:22.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:22.420 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:23.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:23.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:23.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 17 + 1 2026-02-17T06:45:23.423 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=18 2026-02-17T06:45:23.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:23.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:23.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:23.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:23.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:23.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:23.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:24.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:24.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:24.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 18 + 1 2026-02-17T06:45:24.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=19 2026-02-17T06:45:24.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:24.578 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:24.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:24.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:24.579 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:24.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T06:43:22.627468+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:24.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T06:45:25.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T06:45:25.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T06:45:25.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 19 + 1 2026-02-17T06:45:25.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=20 2026-02-17T06:45:25.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T06:45:25.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:25.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:25.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:25.733 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:25.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-17T06:45:23.205923+0000 '>' 2026-02-17T06:44:58.487100+0000 2026-02-17T06:45:25.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:280: TEST_scrub_extended_sleep: '[' 20 -lt 10 ']' 2026-02-17T06:45:25.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:284: TEST_scrub_extended_sleep: PASSED=true 2026-02-17T06:45:25.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:285: TEST_scrub_extended_sleep: break 2026-02-17T06:45:25.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:291: TEST_scrub_extended_sleep: '[' true = false ']' 2026-02-17T06:45:25.884 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_scrub_extended_sleep ------------------ 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_extended_sleep ------------------' 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:45:25.886 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:45:26.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:45:26.194 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:45:26.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:45:26.196 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:45:26.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:45:26.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:45:26.198 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:45:26.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:45:26.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:45:26.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:45:26.200 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:45:26.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:45:26.201 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:45:26.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:45:26.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:45:26.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:45:26.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:45:26.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:45:26.213 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_scrub_extended_sleep ------------------ 2026-02-17T06:45:26.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_extended_sleep ------------------' 2026-02-17T06:45:26.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:45:26.214 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_scrub_permit_time ------------------- 2026-02-17T06:45:26.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_permit_time -------------------' 2026-02-17T06:45:26.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:45:26.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:45:26.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:45:26.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:45:26.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:45:26.215 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:45:26.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:45:26.216 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:45:26.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:45:26.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:45:26.216 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:45:26.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:45:26.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:45:26.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:45:26.217 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:45:26.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:45:26.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:45:26.219 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:45:26.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:45:26.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:45:26.220 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:45:26.221 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:45:26.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:45:26.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:45:26.223 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:45:26.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:45:26.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:45:26.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:45:26.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:45:26.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:45:26.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:45:26.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:45:26.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:45:26.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:45:26.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:45:26.231 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:45:26.231 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_scrub_permit_time ----------------------- 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_permit_time -----------------------' 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_permit_time td/osd-scrub-test 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:419: TEST_scrub_permit_time: local dir=td/osd-scrub-test 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:420: TEST_scrub_permit_time: local poolname=test 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:421: TEST_scrub_permit_time: local OSDS=3 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:422: TEST_scrub_permit_time: local objects=15 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:424: TEST_scrub_permit_time: TESTDATA=testdata.507902 2026-02-17T06:45:26.233 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:426: TEST_scrub_permit_time: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T06:45:26.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:45:26.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:45:26.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:45:26.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:45:26.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:45:26.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T06:45:26.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:45:26.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:45:26.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:45:26.253 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:45:26.253 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.253 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.253 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:26.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T06:45:26.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:45:26.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:45:26.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:45:26.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:45:26.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:45:26.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:45:26.284 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:45:26.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:45:26.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:45:26.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:45:26.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:45:26.358 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:45:26.359 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:45:26.360 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:45:26.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:427: TEST_scrub_permit_time: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:45:26.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:45:26.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:45:26.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:45:26.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:45:26.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:45:26.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:45:26.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:45:26.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:45:26.511 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:45:26.512 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:45:26.512 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.512 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:26.512 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:45:26.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:45:26.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:428: TEST_scrub_permit_time: date -d '2 hour ago' +%H 2026-02-17T06:45:26.531 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:428: TEST_scrub_permit_time: sed 's/^0//' 2026-02-17T06:45:26.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:428: TEST_scrub_permit_time: local scrub_begin_hour=4 2026-02-17T06:45:26.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:429: TEST_scrub_permit_time: date -d '1 hour ago' +%H 2026-02-17T06:45:26.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:429: TEST_scrub_permit_time: sed 's/^0//' 2026-02-17T06:45:26.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:429: TEST_scrub_permit_time: local scrub_end_hour=5 2026-02-17T06:45:26.535 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: expr 3 - 1 2026-02-17T06:45:26.536 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: seq 0 2 2026-02-17T06:45:26.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:45:26.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:432: TEST_scrub_permit_time: run_osd td/osd-scrub-test 0 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 2026-02-17T06:45:26.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:45:26.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:45:26.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:45:26.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:45:26.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:45:26.543 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5' 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:45:26.544 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:45:26.545 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a159c0e0-008a-429a-9ff6-bd5de05ea2fa 2026-02-17T06:45:26.545 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 a159c0e0-008a-429a-9ff6-bd5de05ea2fa 2026-02-17T06:45:26.546 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a159c0e0-008a-429a-9ff6-bd5de05ea2fa' 2026-02-17T06:45:26.546 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:45:26.551 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCGDpRp805+IBAAJm5zqNZKfta0xufeZdPKgQ== 2026-02-17T06:45:26.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCGDpRp805+IBAAJm5zqNZKfta0xufeZdPKgQ=="}' 2026-02-17T06:45:26.552 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a159c0e0-008a-429a-9ff6-bd5de05ea2fa -i td/osd-scrub-test/0/new.json 2026-02-17T06:45:26.641 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:45:26.651 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:45:26.652 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 --mkfs --key AQCGDpRp805+IBAAJm5zqNZKfta0xufeZdPKgQ== --osd-uuid a159c0e0-008a-429a-9ff6-bd5de05ea2fa 2026-02-17T06:45:26.669 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:26.660+0000 7fb5b63468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:26.670 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:26.664+0000 7fb5b63468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:26.671 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:26.664+0000 7fb5b63468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:26.672 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:26.664+0000 7fb5b63468c0 -1 bdev(0x560f13707c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:45:26.672 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:26.664+0000 7fb5b63468c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:45:29.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:45:29.295 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:45:29.296 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:45:29.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:45:29.297 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:45:29.487 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:45:29.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:45:29.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 2026-02-17T06:45:29.488 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:45:29.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:45:29.489 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:45:29.509 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:29.500+0000 7faf1e46a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:29.511 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:29.504+0000 7faf1e46a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:29.512 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:29.504+0000 7faf1e46a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:29.643 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:45:29.795 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:30.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:30.797 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:30.797 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:45:30.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:45:30.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:30.798 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:45:30.950 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:31.220 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:31.212+0000 7faf1e46a8c0 -1 Falling back to public interface 2026-02-17T06:45:31.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:31.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:31.951 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:45:31.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:45:31.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:31.951 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:45:32.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:33.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:33.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:33.103 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:45:33.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:45:33.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:33.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:45:33.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:34.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:34.253 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:34.254 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:45:34.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:45:34.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:34.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:45:34.278 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:34.268+0000 7faf1e46a8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:45:34.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:35.408 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:35.409 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:35.409 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:45:35.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:45:35.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:35.410 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:45:35.558 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4139182544,v1:127.0.0.1:6803/4139182544] [v2:127.0.0.1:6804/4139182544,v1:127.0.0.1:6805/4139182544] exists,up a159c0e0-008a-429a-9ff6-bd5de05ea2fa 2026-02-17T06:45:35.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:45:35.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:45:35.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:45:35.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:45:35.562 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:432: TEST_scrub_permit_time: run_osd td/osd-scrub-test 1 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:45:35.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:45:35.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5' 2026-02-17T06:45:35.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:45:35.566 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:45:35.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=45c4fe0e-b634-4c90-95d1-caf3230f2f07 2026-02-17T06:45:35.566 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 45c4fe0e-b634-4c90-95d1-caf3230f2f07 2026-02-17T06:45:35.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 45c4fe0e-b634-4c90-95d1-caf3230f2f07' 2026-02-17T06:45:35.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:45:35.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCPDpRpolLwIRAAgqUIRx3hNTC1l3WP5OV/dQ== 2026-02-17T06:45:35.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCPDpRpolLwIRAAgqUIRx3hNTC1l3WP5OV/dQ=="}' 2026-02-17T06:45:35.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 45c4fe0e-b634-4c90-95d1-caf3230f2f07 -i td/osd-scrub-test/1/new.json 2026-02-17T06:45:35.729 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:45:35.738 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:45:35.739 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 --mkfs --key AQCPDpRpolLwIRAAgqUIRx3hNTC1l3WP5OV/dQ== --osd-uuid 45c4fe0e-b634-4c90-95d1-caf3230f2f07 2026-02-17T06:45:35.764 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:35.756+0000 7ff0659a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:35.766 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:35.756+0000 7ff0659a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:35.767 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:35.760+0000 7ff0659a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:35.767 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:35.760+0000 7ff0659a68c0 -1 bdev(0x5565648cfc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:45:35.768 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:35.760+0000 7ff0659a68c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:45:38.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:45:38.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:45:38.056 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:45:38.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:45:38.057 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:45:38.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:45:38.251 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:45:38.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 2026-02-17T06:45:38.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:45:38.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:45:38.252 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:45:38.267 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:38.260+0000 7f6ae84328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:38.268 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:38.260+0000 7f6ae84328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:38.269 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:38.260+0000 7f6ae84328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:38.398 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:38.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:45:38.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:39.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:39.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:39.555 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:45:39.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:45:39.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:39.556 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:45:39.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:39.764 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:39.756+0000 7f6ae84328c0 -1 Falling back to public interface 2026-02-17T06:45:40.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:40.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:40.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:45:40.708 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:45:40.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:40.709 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:45:40.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:41.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:41.862 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:41.863 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:45:41.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:45:41.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:41.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:45:42.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:42.719 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:42.712+0000 7f6ae84328c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:45:43.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:43.014 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:43.015 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:45:43.015 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:45:43.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:43.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:45:43.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:44.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:44.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:44.169 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:45:44.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:45:44.170 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:44.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:45:44.321 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1779882078,v1:127.0.0.1:6811/1779882078] [v2:127.0.0.1:6812/1779882078,v1:127.0.0.1:6813/1779882078] exists,up 45c4fe0e-b634-4c90-95d1-caf3230f2f07 2026-02-17T06:45:44.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:45:44.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:45:44.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:45:44.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:45:44.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:432: TEST_scrub_permit_time: run_osd td/osd-scrub-test 2 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 2026-02-17T06:45:44.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:45:44.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:45:44.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:45:44.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:45:44.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:45:44.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:45:44.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:45:44.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:45:44.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:45:44.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:45:44.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:45:44.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:45:44.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:45:44.331 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:45:44.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:45:44.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:45:44.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:45:44.332 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:45:44.332 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:45:44.332 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:45:44.332 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:45:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:45:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:45:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:45:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:45:44.333 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:45:44.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:45:44.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:45:44.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:45:44.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:45:44.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:45:44.334 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5' 2026-02-17T06:45:44.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:45:44.335 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:45:44.335 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=db28fdd4-9fe0-48f5-8a59-eb2e9bd740d3 2026-02-17T06:45:44.336 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 db28fdd4-9fe0-48f5-8a59-eb2e9bd740d3 2026-02-17T06:45:44.337 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 db28fdd4-9fe0-48f5-8a59-eb2e9bd740d3' 2026-02-17T06:45:44.338 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:45:44.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCYDpRpiD2tExAAc5lzMc/qsZOsln7NDUwuKA== 2026-02-17T06:45:44.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCYDpRpiD2tExAAc5lzMc/qsZOsln7NDUwuKA=="}' 2026-02-17T06:45:44.338 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new db28fdd4-9fe0-48f5-8a59-eb2e9bd740d3 -i td/osd-scrub-test/2/new.json 2026-02-17T06:45:44.488 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:45:44.497 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T06:45:44.498 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 --mkfs --key AQCYDpRpiD2tExAAc5lzMc/qsZOsln7NDUwuKA== --osd-uuid db28fdd4-9fe0-48f5-8a59-eb2e9bd740d3 2026-02-17T06:45:44.515 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:44.508+0000 7f30dcae38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:44.517 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:44.508+0000 7f30dcae38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:44.518 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:44.508+0000 7f30dcae38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:44.519 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:44.508+0000 7f30dcae38c0 -1 bdev(0x55efae671c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:45:44.519 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:44.508+0000 7f30dcae38c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T06:45:46.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:45:46.807 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:45:46.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:45:46.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:45:46.999 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:45:47.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:45:47.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=4 --osd_scrub_end_hour=5 2026-02-17T06:45:47.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:45:47.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:45:47.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:45:47.015 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:47.007+0000 7f09e47018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:47.017 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:47.007+0000 7f09e47018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:47.018 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:47.007+0000 7f09e47018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:45:47.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:45:47.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:45:47.150 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:45:47.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:45:47.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:45:47.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:47.151 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:45:47.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:45:47.151 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:47.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:45:47.301 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:48.302 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:45:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:45:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:48.302 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:45:48.452 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:48.728 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:48.719+0000 7f09e47018c0 -1 Falling back to public interface 2026-02-17T06:45:49.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:49.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:49.454 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:45:49.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:45:49.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:49.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:45:49.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:50.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:50.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:50.607 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:45:50.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:45:50.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:50.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:45:50.756 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:51.717 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:45:51.707+0000 7f09e47018c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:45:51.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:51.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:51.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:45:51.758 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:45:51.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:51.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:45:51.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:45:52.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:45:52.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:45:52.914 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:45:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:45:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:45:52.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:45:53.063 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2687499596,v1:127.0.0.1:6819/2687499596] [v2:127.0.0.1:6820/2687499596,v1:127.0.0.1:6821/2687499596] exists,up db28fdd4-9fe0-48f5-8a59-eb2e9bd740d3 2026-02-17T06:45:53.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:45:53.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:45:53.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:45:53.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:440: TEST_scrub_permit_time: create_pool test 1 1 2026-02-17T06:45:53.064 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:45:53.261 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:45:53.272 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:45:54.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:441: TEST_scrub_permit_time: wait_for_clean 2026-02-17T06:45:54.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:45:54.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:45:54.274 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:45:54.275 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:45:54.275 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:45:54.275 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:45:54.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:45:54.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:45:54.276 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:45:54.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:45:54.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:45:54.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:45:54.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:45:54.378 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:45:54.378 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:45:54.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:45:54.533 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:45:54.533 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:45:54.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:45:54.533 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:45:54.533 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:45:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T06:45:54.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T06:45:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T06:45:54.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:45:54.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:45:54.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672972 2026-02-17T06:45:54.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672972 2026-02-17T06:45:54.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672972' 2026-02-17T06:45:54.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:45:54.673 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:45:54.744 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509443 2026-02-17T06:45:54.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509443 2026-02-17T06:45:54.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672972 2-64424509443' 2026-02-17T06:45:54.745 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:45:54.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T06:45:54.745 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:45:54.746 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:45:54.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T06:45:54.747 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:45:54.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T06:45:54.749 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836501 2026-02-17T06:45:54.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T06:45:54.750 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:45:54.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-17T06:45:54.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:45:54.906 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672972 2026-02-17T06:45:54.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:45:54.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:45:54.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672972 2026-02-17T06:45:54.909 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:45:54.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672972 2026-02-17T06:45:54.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672972' 2026-02-17T06:45:54.911 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672972 2026-02-17T06:45:54.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:45:55.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672972 -lt 42949672972 2026-02-17T06:45:55.062 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:45:55.062 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509443 2026-02-17T06:45:55.063 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:45:55.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:45:55.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509443 2026-02-17T06:45:55.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:45:55.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509443 2026-02-17T06:45:55.066 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-17T06:45:55.066 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509443 2026-02-17T06:45:55.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:45:55.219 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-17T06:45:55.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:45:55.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:45:55.220 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:45:55.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:45:55.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:45:55.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:45:55.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:45:55.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:45:55.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:45:55.733 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:45:55.734 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:45:55.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:444: TEST_scrub_permit_time: get_pg test SOMETHING 2026-02-17T06:45:55.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T06:45:55.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T06:45:55.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test SOMETHING 2026-02-17T06:45:55.734 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T06:45:55.883 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:444: TEST_scrub_permit_time: local pgid=1.0 2026-02-17T06:45:55.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:445: TEST_scrub_permit_time: get_primary test SOMETHING 2026-02-17T06:45:55.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:45:55.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T06:45:55.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test SOMETHING 2026-02-17T06:45:55.884 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:45:56.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:445: TEST_scrub_permit_time: local primary=1 2026-02-17T06:45:56.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:446: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:45:56.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:56.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:56.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:56.038 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:56.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:446: TEST_scrub_permit_time: local last_scrub=2026-02-17T06:45:53.254913+0000 2026-02-17T06:45:56.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:451: TEST_scrub_permit_time: ceph tell 1.0 schedule-scrub 86400 2026-02-17T06:45:56.254 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:45:56.254 INFO:tasks.workunit.client.0.trial074.stdout: "deep": false, 2026-02-17T06:45:56.255 INFO:tasks.workunit.client.0.trial074.stdout: "must": false, 2026-02-17T06:45:56.255 INFO:tasks.workunit.client.0.trial074.stdout: "stamp": "2026-02-16T06:44:16.248682+0000" 2026-02-17T06:45:56.255 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:45:56.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i=0 )) 2026-02-17T06:45:56.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:45:56.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:45:56.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:56.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:56.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:56.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:56.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-17T06:45:53.254913+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:45:56.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:45:57.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:45:57.418 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:45:57.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:45:57.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:57.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:57.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:57.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:57.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-17T06:45:53.254913+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:45:57.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:45:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:45:58.570 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:45:58.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:45:58.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:58.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:58.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:58.571 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:58.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-17T06:45:53.254913+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:45:58.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:45:59.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:45:59.723 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:45:59.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:45:59.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:45:59.724 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:45:59.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:45:59.725 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:45:59.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:45:59.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:00.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:00.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:00.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:00.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:00.875 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:00.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:00.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:01.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:01.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:02.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:02.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:02.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:02.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:02.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:02.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:02.028 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:02.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:02.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:03.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:03.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:03.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:03.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:03.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:03.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:03.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:03.329 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:03.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:04.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:04.330 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:04.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:04.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:04.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:04.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:04.331 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:04.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:04.482 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:05.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:05.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:05.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:05.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:05.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:05.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:05.485 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:05.632 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:05.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:06.633 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:06.634 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:06.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:06.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:06.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:06.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:06.635 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:06.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:06.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:07.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:07.786 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:07.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:07.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:07.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:07.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:07.787 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:07.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:07.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:08.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:08.936 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:08.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:08.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:08.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:08.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:08.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:09.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:09.088 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:10.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:10.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:10.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:10.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:10.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:10.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:10.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:10.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:10.237 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:11.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:11.238 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:11.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:11.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:11.239 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:11.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:11.240 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:11.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:11.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:12.395 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:12.396 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:12.396 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:12.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:12.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:12.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:12.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:12.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:12.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:13.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:13.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:13.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:13.548 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:13.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:13.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:13.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:13.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:13.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:14.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:14.703 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:14.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:14.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:14.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:14.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:14.704 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:14.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:14.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:15.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:15.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:15.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:15.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:15.858 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:15.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:15.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:16.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:16.007 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:17.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:17.008 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:17.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:17.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:17.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:17.009 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:17.010 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:17.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:17.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:18.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:18.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:18.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:18.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:18.161 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:18.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:18.162 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:18.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:18.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:19.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:19.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:19.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:19.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:19.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:19.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:19.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:19.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:19.464 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:20.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:20.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:20.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:20.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:20.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:20.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:20.467 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:20.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:20.615 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:21.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:21.616 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:21.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:21.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:21.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:21.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:21.616 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:21.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:21.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:22.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:22.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:22.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:22.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:22.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:22.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:22.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:22.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:22.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:23.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:23.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:23.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:23.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:23.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:23.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:23.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:24.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:24.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:25.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:25.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:25.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:25.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:25.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:25.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:25.073 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:25.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:25.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:26.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:26.224 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:26.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:26.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:26.224 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:26.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:26.225 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:26.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:26.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:27.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:27.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:27.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:27.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:27.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:27.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:27.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:27.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:27.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:28.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:28.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:28.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:28.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:28.528 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:28.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:28.529 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:28.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:28.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:29.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:29.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:29.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T06:46:29.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:46:29.680 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:46:29.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:46:29.681 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:46:29.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T06:44:16.248682+0000 '>' 2026-02-17T06:45:53.254913+0000 2026-02-17T06:46:29.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T06:46:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T06:46:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T06:46:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:460: TEST_scrub_permit_time: perf_counters td/osd-scrub-test 3 2026-02-17T06:46:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:46:30.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T06:46:30.834 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T06:46:30.835 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T06:46:30.836 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:46:30.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:46:30.837 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.902 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.903 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.904 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.905 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.907 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.907 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.907 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:46:30.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:46:30.970 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:46:30.970 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:46:30.970 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.971 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.972 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.973 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.974 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.975 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.976 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:46:30.977 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.978 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.979 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.980 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:30.981 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.982 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:30.983 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:30.984 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:30.985 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:30.985 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:30.985 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.985 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:30.985 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:46:30.985 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:46:30.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:46:30.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T06:46:30.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:46:31.037 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:31.038 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:31.039 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:31.040 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:31.041 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.042 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:31.043 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.044 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:31.045 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.046 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:31.047 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:46:31.048 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.049 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:46:31.050 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:46:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:46:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:46:31.051 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:46:31.052 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:46:31.053 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_scrub_permit_time ------------------ 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_permit_time ------------------' 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:46:31.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:46:31.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:46:31.153 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:46:31.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:46:31.154 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:46:31.154 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:46:31.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:46:31.155 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:46:31.156 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:46:31.157 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:46:31.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:46:31.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:46:31.158 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:46:31.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:46:31.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:46:31.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:46:31.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:46:31.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.172 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:46:31.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:46:31.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:46:31.174 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_scrub_permit_time ------------------ 2026-02-17T06:46:31.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_permit_time ------------------' 2026-02-17T06:46:31.175 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T06:46:31.175 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Prepare Test TEST_scrub_test ------------------- 2026-02-17T06:46:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_test -------------------' 2026-02-17T06:46:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T06:46:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T06:46:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T06:46:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:46:31.177 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:46:31.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:46:31.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:46:31.178 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:46:31.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:46:31.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:46:31.178 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:46:31.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:46:31.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:46:31.179 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:46:31.179 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:46:31.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:46:31.180 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:46:31.180 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:46:31.181 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:46:31.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:46:31.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:46:31.182 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:46:31.183 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:46:31.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:46:31.184 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:46:31.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:46:31.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.186 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.186 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:46:31.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:46:31.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:46:31.187 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T06:46:31.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:46:31.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.189 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.189 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.507902 2026-02-17T06:46:31.190 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:46:31.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T06:46:31.191 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:46:31.192 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T06:46:31.192 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Run Test TEST_scrub_test ----------------------- 2026-02-17T06:46:31.193 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_test -----------------------' 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_test td/osd-scrub-test 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:52: TEST_scrub_test: local dir=td/osd-scrub-test 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:53: TEST_scrub_test: local poolname=test 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:54: TEST_scrub_test: local OSDS=3 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:55: TEST_scrub_test: local objects=15 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:57: TEST_scrub_test: TESTDATA=testdata.507902 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:59: TEST_scrub_test: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T06:46:31.194 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T06:46:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:46:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:46:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:46:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T06:46:31.195 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T06:46:31.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:46:31.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:46:31.213 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:46:31.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:46:31.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.213 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.214 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:31.214 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T06:46:31.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:46:31.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:46:31.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:46:31.242 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:46:31.243 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:46:31.244 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:46:31.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:46:31.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:46:31.244 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:46:31.244 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:46:31.244 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.244 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.245 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:46:31.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:46:31.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get fsid 2026-02-17T06:46:31.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:46:31.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:46:31.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:46:31.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:46:31.301 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:46:31.302 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:46:31.302 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:46:31.302 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:46:31.303 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:46:31.303 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.303 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.303 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.507902/ceph-mon.a.asok 2026-02-17T06:46:31.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:46:31.303 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.507902/ceph-mon.a.asok config get mon_host 2026-02-17T06:46:31.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:60: TEST_scrub_test: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T06:46:31.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T06:46:31.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:46:31.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:46:31.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:46:31.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T06:46:31.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:46:31.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:46:31.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:46:31.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:46:31.449 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:46:31.449 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.449 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.449 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:31.450 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:46:31.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T06:46:31.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:61: TEST_scrub_test: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T06:46:31.467 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:62: TEST_scrub_test: ceph_osd_args+='--osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 ' 2026-02-17T06:46:31.468 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:63: TEST_scrub_test: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2026-02-17T06:46:31.468 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: expr 3 - 1 2026-02-17T06:46:31.469 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: seq 0 2 2026-02-17T06:46:31.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:46:31.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:66: TEST_scrub_test: run_osd td/osd-scrub-test 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:46:31.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:46:31.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:46:31.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:46:31.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:46:31.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:46:31.473 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:46:31.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:46:31.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:46:31.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:46:31.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T06:46:31.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T06:46:31.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:46:31.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8bd1de99-3bba-4dac-9e7c-2740da680399 2026-02-17T06:46:31.476 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 8bd1de99-3bba-4dac-9e7c-2740da680399 2026-02-17T06:46:31.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 8bd1de99-3bba-4dac-9e7c-2740da680399' 2026-02-17T06:46:31.477 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:46:31.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDHDpRpwa11HBAAGI8tZmkOd11QdHE8Q+gKCg== 2026-02-17T06:46:31.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDHDpRpwa11HBAAGI8tZmkOd11QdHE8Q+gKCg=="}' 2026-02-17T06:46:31.484 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8bd1de99-3bba-4dac-9e7c-2740da680399 -i td/osd-scrub-test/0/new.json 2026-02-17T06:46:31.571 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:46:31.579 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T06:46:31.580 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQDHDpRpwa11HBAAGI8tZmkOd11QdHE8Q+gKCg== --osd-uuid 8bd1de99-3bba-4dac-9e7c-2740da680399 2026-02-17T06:46:31.596 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:31.587+0000 7f56af68e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:31.597 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:31.587+0000 7f56af68e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:31.598 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:31.587+0000 7f56af68e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:31.599 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:31.591+0000 7f56af68e8c0 -1 bdev(0x55ee4e41fc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:46:31.599 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:31.591+0000 7f56af68e8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T06:46:34.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T06:46:34.196 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:46:34.197 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:46:34.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:46:34.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:46:34.390 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:46:34.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:46:34.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:46:34.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:46:34.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:46:34.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:46:34.415 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:34.407+0000 7f1a8de058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:34.417 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:34.407+0000 7f1a8de058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:34.418 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:34.407+0000 7f1a8de058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:34.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:46:34.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:46:34.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:46:34.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:46:34.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:46:34.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:34.540 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:46:34.540 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:46:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:34.541 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:46:34.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:35.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:35.694 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:35.694 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:46:35.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:46:35.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:35.695 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:46:35.846 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:35.920 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:35.911+0000 7f1a8de058c0 -1 Falling back to public interface 2026-02-17T06:46:36.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:36.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:36.848 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:46:36.849 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:46:36.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:36.849 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:46:37.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:38.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:38.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:38.006 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:46:38.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:46:38.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:38.006 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:46:38.158 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:38.903 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:38.895+0000 7f1a8de058c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:46:39.159 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:39.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:39.160 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:46:39.160 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:46:39.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:46:39.161 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:39.313 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:40.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:40.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:40.315 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:46:40.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:46:40.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:40.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:46:40.469 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2688765708,v1:127.0.0.1:6803/2688765708] [v2:127.0.0.1:6804/2688765708,v1:127.0.0.1:6805/2688765708] exists,up 8bd1de99-3bba-4dac-9e7c-2740da680399 2026-02-17T06:46:40.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:46:40.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:46:40.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:46:40.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:46:40.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:66: TEST_scrub_test: run_osd td/osd-scrub-test 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:46:40.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:46:40.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:46:40.475 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T06:46:40.476 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:46:40.477 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=48bf86f1-7456-4f0c-ad1b-7f1733b98571 2026-02-17T06:46:40.477 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 48bf86f1-7456-4f0c-ad1b-7f1733b98571 2026-02-17T06:46:40.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 48bf86f1-7456-4f0c-ad1b-7f1733b98571' 2026-02-17T06:46:40.478 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:46:40.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDQDpRpVOqCHBAA3SUoa0Y17Krml46PXlRb3g== 2026-02-17T06:46:40.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDQDpRpVOqCHBAA3SUoa0Y17Krml46PXlRb3g=="}' 2026-02-17T06:46:40.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 48bf86f1-7456-4f0c-ad1b-7f1733b98571 -i td/osd-scrub-test/1/new.json 2026-02-17T06:46:40.636 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:46:40.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T06:46:40.646 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQDQDpRpVOqCHBAA3SUoa0Y17Krml46PXlRb3g== --osd-uuid 48bf86f1-7456-4f0c-ad1b-7f1733b98571 2026-02-17T06:46:40.663 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:40.655+0000 7f89a2a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:40.665 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:40.655+0000 7f89a2a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:40.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:40.655+0000 7f89a2a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:40.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:40.655+0000 7f89a2a658c0 -1 bdev(0x55cd6f8e1c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:46:40.666 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:40.659+0000 7f89a2a658c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T06:46:42.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T06:46:42.940 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:46:42.941 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:46:42.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:46:42.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:46:43.135 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:46:43.136 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:46:43.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:46:43.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:46:43.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:46:43.137 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:46:43.159 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:43.151+0000 7f7620dd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:43.161 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:43.151+0000 7f7620dd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:43.162 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:43.151+0000 7f7620dd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:43.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:46:43.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:46:43.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:46:43.289 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:46:43.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:46:43.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:43.290 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:46:43.290 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:46:43.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:43.291 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:46:43.440 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:44.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:44.441 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:44.441 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:46:44.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:46:44.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:44.442 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:46:44.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:44.652 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:44.643+0000 7f7620dd98c0 -1 Falling back to public interface 2026-02-17T06:46:45.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:45.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:45.599 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:46:45.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:46:45.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:45.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:46:45.749 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:46.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:46.750 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:46.751 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:46:46.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:46:46.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:46.752 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:46:46.900 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:47.670 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:47.659+0000 7f7620dd98c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:46:47.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:47.901 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:47.902 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:46:47.902 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:46:47.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:47.903 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:46:48.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:49.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:49.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:49.053 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:46:49.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:46:49.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:49.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:46:49.205 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3648653692,v1:127.0.0.1:6811/3648653692] [v2:127.0.0.1:6812/3648653692,v1:127.0.0.1:6813/3648653692] exists,up 48bf86f1-7456-4f0c-ad1b-7f1733b98571 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:66: TEST_scrub_test: run_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:46:49.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:46:49.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:46:49.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:46:49.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:46:49.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:46:49.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:46:49.209 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:46:49.209 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:46:49.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:46:49.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:46:49.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:46:49.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T06:46:49.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:46:49.211 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:46:49.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=eaa5ef61-420c-46cd-a15a-e508a39d4f68 2026-02-17T06:46:49.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 eaa5ef61-420c-46cd-a15a-e508a39d4f68' 2026-02-17T06:46:49.212 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 eaa5ef61-420c-46cd-a15a-e508a39d4f68 2026-02-17T06:46:49.212 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:46:49.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDZDpRpSmPVDBAAr+Xkfng9ISKQNSZteD2gjA== 2026-02-17T06:46:49.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDZDpRpSmPVDBAAr+Xkfng9ISKQNSZteD2gjA=="}' 2026-02-17T06:46:49.222 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new eaa5ef61-420c-46cd-a15a-e508a39d4f68 -i td/osd-scrub-test/2/new.json 2026-02-17T06:46:49.376 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:46:49.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T06:46:49.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQDZDpRpSmPVDBAAr+Xkfng9ISKQNSZteD2gjA== --osd-uuid eaa5ef61-420c-46cd-a15a-e508a39d4f68 2026-02-17T06:46:49.403 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:49.395+0000 7fd2535678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:49.405 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:49.395+0000 7fd2535678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:49.406 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:49.395+0000 7fd2535678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:49.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:49.395+0000 7fd2535678c0 -1 bdev(0x55c7897e7c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:46:49.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:49.399+0000 7fd2535678c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T06:46:51.672 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T06:46:51.673 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:46:51.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:46:51.674 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:46:51.675 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:46:51.869 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:46:51.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:46:51.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:46:51.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:46:51.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:46:51.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:46:51.893 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:51.883+0000 7f5dfb8238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:51.895 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:51.887+0000 7f5dfb8238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:51.897 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:51.887+0000 7f5dfb8238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:46:52.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:52.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:46:52.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:53.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:53.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:53.174 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:46:53.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:46:53.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:53.174 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:46:53.325 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:53.400 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:53.391+0000 7f5dfb8238c0 -1 Falling back to public interface 2026-02-17T06:46:54.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:54.326 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:54.327 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:46:54.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:46:54.327 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:54.328 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:46:54.478 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:55.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:55.480 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:55.480 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:46:55.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:46:55.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:55.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:46:55.628 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:56.414 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:46:56.403+0000 7f5dfb8238c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:46:56.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:56.630 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:56.630 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:46:56.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:46:56.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:56.631 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:46:56.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:46:57.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:46:57.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:46:57.780 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:46:57.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:46:57.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:46:57.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:46:57.933 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/262612421,v1:127.0.0.1:6819/262612421] [v2:127.0.0.1:6820/262612421,v1:127.0.0.1:6821/262612421] exists,up eaa5ef61-420c-46cd-a15a-e508a39d4f68 2026-02-17T06:46:57.934 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:46:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:46:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:46:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:70: TEST_scrub_test: create_pool test 1 1 2026-02-17T06:46:57.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T06:46:58.129 INFO:tasks.workunit.client.0.trial074.stderr:pool 'test' created 2026-02-17T06:46:58.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T06:46:59.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:71: TEST_scrub_test: wait_for_clean 2026-02-17T06:46:59.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:46:59.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:46:59.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:46:59.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:46:59.143 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:46:59.143 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:46:59.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:46:59.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:46:59.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:46:59.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:46:59.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:46:59.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:46:59.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:46:59.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:46:59.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:46:59.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:46:59.394 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:46:59.394 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:46:59.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:46:59.394 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:46:59.394 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:46:59.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T06:46:59.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T06:46:59.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T06:46:59.466 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:46:59.466 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:46:59.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705676 2026-02-17T06:46:59.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705676 2026-02-17T06:46:59.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676' 2026-02-17T06:46:59.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:46:59.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:46:59.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542149 2026-02-17T06:46:59.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542149 2026-02-17T06:46:59.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676 2-60129542149' 2026-02-17T06:46:59.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:46:59.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T06:46:59.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:46:59.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:46:59.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T06:46:59.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:46:59.608 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T06:46:59.609 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836501 2026-02-17T06:46:59.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T06:46:59.609 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:46:59.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-17T06:46:59.770 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:46:59.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705676 2026-02-17T06:46:59.771 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:46:59.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:46:59.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705676 2026-02-17T06:46:59.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:46:59.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705676 2026-02-17T06:46:59.774 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705676 2026-02-17T06:46:59.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-17T06:46:59.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:46:59.932 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705676 -lt 38654705676 2026-02-17T06:46:59.933 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:46:59.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542149 2026-02-17T06:46:59.933 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:46:59.935 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:46:59.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542149 2026-02-17T06:46:59.935 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:46:59.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542149 2026-02-17T06:46:59.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-17T06:46:59.937 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 60129542149 2026-02-17T06:46:59.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:00.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542149 -lt 60129542149 2026-02-17T06:47:00.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:47:00.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:00.101 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:00.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:47:00.282 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:47:00.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:47:00.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:47:00.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:47:00.282 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:47:00.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:47:00.283 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:47:00.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:47:00.432 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:47:00.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:00.433 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:00.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:47:00.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:47:00.613 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:47:00.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: ceph osd dump 2026-02-17T06:47:00.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: grep '^pool.*['\'']test['\'']' 2026-02-17T06:47:00.614 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: awk '{ print $2 }' 2026-02-17T06:47:00.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: poolid=1 2026-02-17T06:47:00.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:74: TEST_scrub_test: dd if=/dev/urandom of=testdata.507902 bs=1032 count=1 2026-02-17T06:47:00.765 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:47:00.765 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:47:00.765 INFO:tasks.workunit.client.0.trial074.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.7086e-05 s, 18.1 MB/s 2026-02-17T06:47:00.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: seq 1 15 2026-02-17T06:47:00.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.766 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj1 testdata.507902 2026-02-17T06:47:00.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.785 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj2 testdata.507902 2026-02-17T06:47:00.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.806 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj3 testdata.507902 2026-02-17T06:47:00.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.824 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj4 testdata.507902 2026-02-17T06:47:00.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.843 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj5 testdata.507902 2026-02-17T06:47:00.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.865 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj6 testdata.507902 2026-02-17T06:47:00.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj7 testdata.507902 2026-02-17T06:47:00.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj8 testdata.507902 2026-02-17T06:47:00.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.939 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj9 testdata.507902 2026-02-17T06:47:00.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.963 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj10 testdata.507902 2026-02-17T06:47:00.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:00.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj11 testdata.507902 2026-02-17T06:47:01.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:01.012 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj12 testdata.507902 2026-02-17T06:47:01.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:01.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj13 testdata.507902 2026-02-17T06:47:01.060 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:01.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj14 testdata.507902 2026-02-17T06:47:01.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T06:47:01.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj15 testdata.507902 2026-02-17T06:47:01.109 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:79: TEST_scrub_test: rm -f testdata.507902 2026-02-17T06:47:01.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:81: TEST_scrub_test: get_primary test obj1 2026-02-17T06:47:01.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:47:01.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:47:01.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:47:01.112 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:47:01.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:81: TEST_scrub_test: local primary=1 2026-02-17T06:47:01.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:82: TEST_scrub_test: get_not_primary test obj1 2026-02-17T06:47:01.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=test 2026-02-17T06:47:01.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T06:47:01.260 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary test obj1 2026-02-17T06:47:01.260 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T06:47:01.260 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T06:47:01.261 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T06:47:01.261 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:47:01.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T06:47:01.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map test obj1 2026-02-17T06:47:01.416 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T06:47:01.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:82: TEST_scrub_test: local otherosd=0 2026-02-17T06:47:01.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:83: TEST_scrub_test: '[' 0 = 2 ']' 2026-02-17T06:47:01.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:87: TEST_scrub_test: local anotherosd=2 2026-02-17T06:47:01.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:90: TEST_scrub_test: CORRUPT_DATA=corrupt-data.507902 2026-02-17T06:47:01.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:91: TEST_scrub_test: dd if=/dev/urandom of=corrupt-data.507902 bs=512 count=1 2026-02-17T06:47:01.566 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records in 2026-02-17T06:47:01.566 INFO:tasks.workunit.client.0.trial074.stderr:1+0 records out 2026-02-17T06:47:01.566 INFO:tasks.workunit.client.0.trial074.stderr:512 bytes copied, 5.1947e-05 s, 9.9 MB/s 2026-02-17T06:47:01.566 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:92: TEST_scrub_test: objectstore_tool td/osd-scrub-test 2 obj1 set-bytes corrupt-data.507902 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-test 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-test 2 obj1 set-bytes corrupt-data.507902 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-test 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-test TERM osd.2 2026-02-17T06:47:01.567 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:47:01.568 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:47:01.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:47:01.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:47:01.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:47:02.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:47:02.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-test 2 obj1 set-bytes corrupt-data.507902 2026-02-17T06:47:02.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-test 2026-02-17T06:47:02.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:47:02.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:47:02.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:47:02.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-test/2 2026-02-17T06:47:02.876 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-test/2 obj1 set-bytes corrupt-data.507902 2026-02-17T06:47:04.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-test 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T06:47:04.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:47:04.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:47:04.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' 2026-02-17T06:47:04.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T06:47:04.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:47:04.084 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:47:04.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=297b5593-5c9b-487d-8ca3-03d1a6f8eb85 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.507902/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T06:47:04.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-test/2/whoami 2026-02-17T06:47:04.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:47:04.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:47:04.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:47:04.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:47:04.104 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:04.094+0000 7f2bb3ff68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:04.105 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:04.094+0000 7f2bb3ff68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:04.106 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:04.094+0000 7f2bb3ff68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:47:04.225 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:47:04.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:04.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:47:04.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:05.340 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:05.330+0000 7f2bb3ff68c0 -1 Falling back to public interface 2026-02-17T06:47:05.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:05.380 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:47:05.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:47:05.381 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:05.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:05.381 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:47:05.528 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:06.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:06.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:47:06.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:47:06.530 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:47:06.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:06.531 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:47:06.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:07.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:07.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:47:07.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:47:07.681 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:47:07.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:07.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:47:07.831 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:08.306 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:08.294+0000 7f2bb3ff68c0 -1 osd.2 18 log_to_monitors true 2026-02-17T06:47:08.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:08.832 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T06:47:08.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:47:08.833 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:47:08.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:08.833 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 22 up_thru 0 down_at 19 last_clean_interval [14,18) [v2:127.0.0.1:6818/801681820,v1:127.0.0.1:6819/801681820] [v2:127.0.0.1:6820/801681820,v1:127.0.0.1:6821/801681820] exists,up eaa5ef61-420c-46cd-a15a-e508a39d4f68 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:47:08.979 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:47:08.980 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:47:08.981 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:47:08.981 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:47:08.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:47:08.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:47:08.982 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:47:09.090 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:47:09.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:47:09.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:47:09.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:47:09.091 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:47:09.091 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:47:09.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:47:09.244 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:09.244 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:47:09.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:47:09.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:09.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:47:09.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836512 2026-02-17T06:47:09.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836512 2026-02-17T06:47:09.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512' 2026-02-17T06:47:09.315 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:09.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:47:09.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705687 2026-02-17T06:47:09.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705687 2026-02-17T06:47:09.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-38654705687' 2026-02-17T06:47:09.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:09.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:47:09.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280515 2026-02-17T06:47:09.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280515 2026-02-17T06:47:09.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-38654705687 2-94489280515' 2026-02-17T06:47:09.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:09.454 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836512 2026-02-17T06:47:09.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:09.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:47:09.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836512 2026-02-17T06:47:09.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:09.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836512 2026-02-17T06:47:09.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836512' 2026-02-17T06:47:09.458 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836512 2026-02-17T06:47:09.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:09.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836509 -lt 21474836512 2026-02-17T06:47:09.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:10.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:47:10.607 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:10.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836512 -lt 21474836512 2026-02-17T06:47:10.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:10.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705687 2026-02-17T06:47:10.758 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:10.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:47:10.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705687 2026-02-17T06:47:10.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:10.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705687 2026-02-17T06:47:10.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705687' 2026-02-17T06:47:10.762 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 38654705687 2026-02-17T06:47:10.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:47:10.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705687 -lt 38654705687 2026-02-17T06:47:10.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:10.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-94489280515 2026-02-17T06:47:10.926 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:10.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:47:10.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-94489280515 2026-02-17T06:47:10.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:10.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280515 2026-02-17T06:47:10.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 94489280515' 2026-02-17T06:47:10.930 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 94489280515 2026-02-17T06:47:10.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:11.089 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280515 -lt 94489280515 2026-02-17T06:47:11.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:47:11.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:11.090 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:11.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:47:11.270 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:47:11.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:47:11.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:47:11.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:47:11.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:47:11.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:47:11.271 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:47:11.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:47:11.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:47:11.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:11.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:11.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:47:11.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:47:11.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:47:11.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:93: TEST_scrub_test: rm -f corrupt-data.507902 2026-02-17T06:47:11.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:95: TEST_scrub_test: local pgid=1.0 2026-02-17T06:47:11.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:96: TEST_scrub_test: pg_deep_scrub 1.0 2026-02-17T06:47:11.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=1.0 2026-02-17T06:47:11.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 1.0 2026-02-17T06:47:11.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T06:47:11.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:47:11.601 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:47:11.601 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:47:11.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:47:11.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:47:11.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:47:11.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:47:11.880 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:47:11.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:47:11.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:47:11.881 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:47:11.881 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:47:12.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:47:12.033 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:12.033 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:47:12.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:47:12.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:12.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:47:12.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836516 2026-02-17T06:47:12.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836516 2026-02-17T06:47:12.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516' 2026-02-17T06:47:12.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:12.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:47:12.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705691 2026-02-17T06:47:12.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705691 2026-02-17T06:47:12.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-38654705691' 2026-02-17T06:47:12.173 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:12.173 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:47:12.244 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280518 2026-02-17T06:47:12.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280518 2026-02-17T06:47:12.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-38654705691 2-94489280518' 2026-02-17T06:47:12.245 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:12.245 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836516 2026-02-17T06:47:12.246 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:12.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:47:12.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836516 2026-02-17T06:47:12.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:12.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836516 2026-02-17T06:47:12.249 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836516 2026-02-17T06:47:12.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836516' 2026-02-17T06:47:12.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:12.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836514 -lt 21474836516 2026-02-17T06:47:12.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:13.402 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:47:13.402 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:13.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836514 -lt 21474836516 2026-02-17T06:47:13.553 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:14.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T06:47:14.554 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:14.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836516 2026-02-17T06:47:14.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:14.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705691 2026-02-17T06:47:14.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:14.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:47:14.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705691 2026-02-17T06:47:14.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:14.710 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705691 2026-02-17T06:47:14.710 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705691 2026-02-17T06:47:14.711 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705691' 2026-02-17T06:47:14.711 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:47:14.873 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705692 -lt 38654705691 2026-02-17T06:47:14.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:14.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-94489280518 2026-02-17T06:47:14.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:14.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:47:14.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-94489280518 2026-02-17T06:47:14.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:14.877 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280518 2026-02-17T06:47:14.878 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 94489280518 2026-02-17T06:47:14.878 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 94489280518' 2026-02-17T06:47:14.878 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:15.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280520 -lt 94489280518 2026-02-17T06:47:15.039 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:47:15.040 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 1.0 loop 0 2026-02-17T06:47:15.040 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T06:47:15.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T06:47:15.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T06:47:15.041 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:47:15.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T06:47:15.041 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:47:15.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:15.111 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:15.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T06:46:58.122559+0000 2026-02-17T06:47:15.254 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 1.0 2026-02-17T06:47:15.397 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2026-02-17T06:47:15.406 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 1.0 2026-02-17T06:46:58.122559+0000 last_deep_scrub_stamp 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:46:58.122559+0000 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:15.407 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:15.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:15.408 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:15.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:46:58.122559+0000 '>' 2026-02-17T06:46:58.122559+0000 2026-02-17T06:47:15.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:47:16.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:47:16.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:16.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:16.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:16.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:16.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:16.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:16.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:46:58.122559+0000 '>' 2026-02-17T06:46:58.122559+0000 2026-02-17T06:47:16.707 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:47:17.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:47:17.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:17.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:17.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:17.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:17.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:17.709 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:17.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:47:15.435049+0000 '>' 2026-02-17T06:46:58.122559+0000 2026-02-17T06:47:17.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:47:17.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:98: TEST_scrub_test: ceph pg dump pgs 2026-02-17T06:47:17.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:98: TEST_scrub_test: grep '^1.0' 2026-02-17T06:47:17.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:98: TEST_scrub_test: grep -q -- +inconsistent 2026-02-17T06:47:18.004 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:47:18.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:99: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T06:47:18.014 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:99: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-17T06:47:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:99: TEST_scrub_test: test 2 = 2 2026-02-17T06:47:18.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:101: TEST_scrub_test: ceph osd out 1 2026-02-17T06:47:18.281 INFO:tasks.workunit.client.0.trial074.stderr:marked out osd.1. 2026-02-17T06:47:18.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:102: TEST_scrub_test: wait_for_clean 2026-02-17T06:47:18.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:47:18.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:47:18.293 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:47:18.293 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:47:18.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:47:18.294 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:47:18.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:47:18.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:47:18.295 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:47:18.411 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:47:18.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:47:18.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:47:18.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:47:18.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:47:18.412 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:47:18.563 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:47:18.563 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:18.563 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:47:18.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:47:18.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:18.564 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:47:18.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836523 2026-02-17T06:47:18.635 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836523 2026-02-17T06:47:18.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523' 2026-02-17T06:47:18.636 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:18.636 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:47:18.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705698 2026-02-17T06:47:18.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705698 2026-02-17T06:47:18.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705698' 2026-02-17T06:47:18.708 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:18.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:47:18.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280526 2026-02-17T06:47:18.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280526 2026-02-17T06:47:18.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836523 1-38654705698 2-94489280526' 2026-02-17T06:47:18.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:18.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836523 2026-02-17T06:47:18.781 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:18.781 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:47:18.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836523 2026-02-17T06:47:18.782 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:18.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836523 2026-02-17T06:47:18.784 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836523 2026-02-17T06:47:18.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836523' 2026-02-17T06:47:18.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:18.941 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836521 -lt 21474836523 2026-02-17T06:47:18.942 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:19.943 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:47:19.943 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:20.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836523 2026-02-17T06:47:20.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:20.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705698 2026-02-17T06:47:20.095 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:20.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:47:20.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705698 2026-02-17T06:47:20.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:20.098 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705698 2026-02-17T06:47:20.098 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705698 2026-02-17T06:47:20.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705698' 2026-02-17T06:47:20.099 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:47:20.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705699 -lt 38654705698 2026-02-17T06:47:20.258 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:20.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-94489280526 2026-02-17T06:47:20.259 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:20.260 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:47:20.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-94489280526 2026-02-17T06:47:20.261 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:20.262 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280526 2026-02-17T06:47:20.262 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 94489280526 2026-02-17T06:47:20.263 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 94489280526' 2026-02-17T06:47:20.263 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:20.422 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280527 -lt 94489280526 2026-02-17T06:47:20.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:47:20.423 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:20.424 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:20.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:47:20.603 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:47:20.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:47:20.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:47:20.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:47:20.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:47:20.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:47:20.605 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:47:20.755 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:47:20.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:47:20.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:20.756 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:104: TEST_scrub_test: pg_deep_scrub 1.0 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=1.0 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 1.0 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T06:47:20.937 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T06:47:20.938 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:47:20.938 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:47:20.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:47:20.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:47:20.938 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:47:21.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T06:47:21.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T06:47:21.217 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T06:47:21.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T06:47:21.218 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:47:21.218 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:47:21.372 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:47:21.373 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:21.373 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:47:21.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:47:21.373 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:21.373 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:47:21.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836527 2026-02-17T06:47:21.443 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836527 2026-02-17T06:47:21.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527' 2026-02-17T06:47:21.444 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:21.444 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:47:21.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705702 2026-02-17T06:47:21.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705702 2026-02-17T06:47:21.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705702' 2026-02-17T06:47:21.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:21.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:47:21.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280530 2026-02-17T06:47:21.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280530 2026-02-17T06:47:21.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-38654705702 2-94489280530' 2026-02-17T06:47:21.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:21.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836527 2026-02-17T06:47:21.585 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:21.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:47:21.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836527 2026-02-17T06:47:21.586 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:21.588 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836527 2026-02-17T06:47:21.588 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836527 2026-02-17T06:47:21.589 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836527' 2026-02-17T06:47:21.589 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:21.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836527 2026-02-17T06:47:21.735 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:22.737 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:47:22.737 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:22.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836527 -lt 21474836527 2026-02-17T06:47:22.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:22.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705702 2026-02-17T06:47:22.888 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:22.888 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:47:22.889 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705702 2026-02-17T06:47:22.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:22.891 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705702 2026-02-17T06:47:22.891 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705702 2026-02-17T06:47:22.892 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705702' 2026-02-17T06:47:22.892 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:47:23.051 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705702 2026-02-17T06:47:23.052 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:23.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-94489280530 2026-02-17T06:47:23.052 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:23.053 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:47:23.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-94489280530 2026-02-17T06:47:23.054 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:23.055 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280530 2026-02-17T06:47:23.056 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 94489280530 2026-02-17T06:47:23.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 94489280530' 2026-02-17T06:47:23.056 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:23.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280530 -lt 94489280530 2026-02-17T06:47:23.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T06:47:23.207 INFO:tasks.workunit.client.0.trial074.stdout:#---------- 1.0 loop 0 2026-02-17T06:47:23.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T06:47:23.208 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T06:47:23.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T06:47:23.209 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T06:47:23.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T06:47:23.209 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T06:47:23.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+remapped 2026-02-17T06:47:23.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+remapped == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T06:47:23.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T06:47:23.278 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T06:47:23.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:23.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:23.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:23.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:23.279 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:23.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T06:47:15.435049+0000 2026-02-17T06:47:23.425 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 1.0 2026-02-17T06:47:23.566 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.0 to deep-scrub 2026-02-17T06:47:23.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 1.0 2026-02-17T06:47:15.435049+0000 last_deep_scrub_stamp 2026-02-17T06:47:23.576 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:47:15.435049+0000 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:23.577 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:23.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:47:15.435049+0000 '>' 2026-02-17T06:47:15.435049+0000 2026-02-17T06:47:23.724 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:47:24.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:47:24.726 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:24.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:24.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:24.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:24.727 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:24.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:24.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:47:15.435049+0000 '>' 2026-02-17T06:47:15.435049+0000 2026-02-17T06:47:24.874 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:47:25.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:47:25.875 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:25.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T06:47:25.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:25.876 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T06:47:25.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:25.877 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T06:47:26.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:47:23.751796+0000 '>' 2026-02-17T06:47:15.435049+0000 2026-02-17T06:47:26.026 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:47:26.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:106: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T06:47:26.027 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:106: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-17T06:47:26.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:106: TEST_scrub_test: test 2 = 2 2026-02-17T06:47:26.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:107: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T06:47:26.097 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:107: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-17T06:47:26.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:107: TEST_scrub_test: test 2 = 2 2026-02-17T06:47:26.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:108: TEST_scrub_test: ceph pg dump pgs 2026-02-17T06:47:26.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:108: TEST_scrub_test: grep '^1.0' 2026-02-17T06:47:26.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:108: TEST_scrub_test: grep -q -- +inconsistent 2026-02-17T06:47:26.305 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:47:26.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:110: TEST_scrub_test: ceph osd in 1 2026-02-17T06:47:26.511 INFO:tasks.workunit.client.0.trial074.stderr:marked in osd.1. 2026-02-17T06:47:26.521 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:111: TEST_scrub_test: wait_for_clean 2026-02-17T06:47:26.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:47:26.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:47:26.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:47:26.522 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:47:26.523 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:47:26.523 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:47:26.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:47:26.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:47:26.523 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:47:26.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:47:26.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:47:26.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:47:26.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:47:26.641 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:47:26.641 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:47:26.788 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:47:26.788 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:26.789 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:47:26.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:47:26.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:26.789 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:47:26.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836533 2026-02-17T06:47:26.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836533 2026-02-17T06:47:26.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533' 2026-02-17T06:47:26.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:26.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:47:26.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705709 2026-02-17T06:47:26.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705709 2026-02-17T06:47:26.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705709' 2026-02-17T06:47:26.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:26.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:47:27.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280536 2026-02-17T06:47:27.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280536 2026-02-17T06:47:27.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-38654705709 2-94489280536' 2026-02-17T06:47:27.000 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:27.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836533 2026-02-17T06:47:27.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:27.002 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:47:27.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836533 2026-02-17T06:47:27.003 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:27.004 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836533 2026-02-17T06:47:27.004 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836533 2026-02-17T06:47:27.005 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836533' 2026-02-17T06:47:27.005 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:27.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836531 -lt 21474836533 2026-02-17T06:47:27.162 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:28.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:47:28.164 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:28.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836533 2026-02-17T06:47:28.314 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:28.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705709 2026-02-17T06:47:28.315 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:28.316 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:47:28.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705709 2026-02-17T06:47:28.317 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:28.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705709 2026-02-17T06:47:28.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705709' 2026-02-17T06:47:28.319 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705709 2026-02-17T06:47:28.319 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:47:28.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705709 -lt 38654705709 2026-02-17T06:47:28.481 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:28.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-94489280536 2026-02-17T06:47:28.482 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:28.483 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:47:28.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-94489280536 2026-02-17T06:47:28.484 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:28.485 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280536 2026-02-17T06:47:28.485 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 94489280536 2026-02-17T06:47:28.486 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 94489280536' 2026-02-17T06:47:28.486 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:28.637 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280537 -lt 94489280536 2026-02-17T06:47:28.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:47:28.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:28.638 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:28.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:47:28.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:47:28.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:47:28.821 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:47:28.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:47:28.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:47:28.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:47:28.822 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:47:28.968 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:47:28.968 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:47:28.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:28.969 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:113: TEST_scrub_test: repair 1.0 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:29.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:47:29.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:29.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:47:29.307 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T06:47:23.751796+0000 2026-02-17T06:47:29.308 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T06:47:29.447 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T06:47:29.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T06:47:23.751796+0000 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T06:47:23.751796+0000 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:29.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:47:29.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:47:23.751796+0000 '>' 2026-02-17T06:47:23.751796+0000 2026-02-17T06:47:29.607 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:47:30.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:47:30.609 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:30.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:47:30.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:30.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:47:30.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:30.611 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:47:30.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:47:23.751796+0000 '>' 2026-02-17T06:47:23.751796+0000 2026-02-17T06:47:30.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T06:47:31.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T06:47:31.760 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T06:47:31.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T06:47:31.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T06:47:31.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T06:47:31.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T06:47:31.761 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T06:47:31.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T06:47:29.579034+0000 '>' 2026-02-17T06:47:23.751796+0000 2026-02-17T06:47:31.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T06:47:31.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:114: TEST_scrub_test: wait_for_clean 2026-02-17T06:47:31.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:47:31.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:47:31.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:47:31.910 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:47:31.911 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:47:31.911 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:47:31.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:47:31.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:47:31.911 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:47:32.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:47:32.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:47:32.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:47:32.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:47:32.017 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:47:32.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:47:32.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:47:32.167 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:32.168 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:47:32.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:47:32.168 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:32.168 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:47:32.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836540 2026-02-17T06:47:32.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836540 2026-02-17T06:47:32.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540' 2026-02-17T06:47:32.241 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:32.241 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:47:32.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705715 2026-02-17T06:47:32.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705715 2026-02-17T06:47:32.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540 1-38654705715' 2026-02-17T06:47:32.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:32.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:47:32.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280542 2026-02-17T06:47:32.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280542 2026-02-17T06:47:32.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540 1-38654705715 2-94489280542' 2026-02-17T06:47:32.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:32.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836540 2026-02-17T06:47:32.387 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:32.387 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:47:32.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836540 2026-02-17T06:47:32.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:32.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836540 2026-02-17T06:47:32.390 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836540' 2026-02-17T06:47:32.390 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836540 2026-02-17T06:47:32.390 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:32.547 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836538 -lt 21474836540 2026-02-17T06:47:32.548 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:33.549 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:47:33.549 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:33.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836538 -lt 21474836540 2026-02-17T06:47:33.701 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:34.702 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T06:47:34.703 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:34.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836541 -lt 21474836540 2026-02-17T06:47:34.856 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:34.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705715 2026-02-17T06:47:34.857 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:34.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:47:34.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705715 2026-02-17T06:47:34.859 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:34.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705715 2026-02-17T06:47:34.860 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705715 2026-02-17T06:47:34.861 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705715' 2026-02-17T06:47:34.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:47:35.020 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705716 -lt 38654705715 2026-02-17T06:47:35.021 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:35.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-94489280542 2026-02-17T06:47:35.021 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:35.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:47:35.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-94489280542 2026-02-17T06:47:35.023 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:35.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280542 2026-02-17T06:47:35.025 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 94489280542' 2026-02-17T06:47:35.025 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 94489280542 2026-02-17T06:47:35.025 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:35.182 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280544 -lt 94489280542 2026-02-17T06:47:35.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:47:35.183 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:35.184 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:35.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:47:35.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:47:35.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:47:35.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:47:35.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:47:35.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:47:35.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:47:35.366 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:47:35.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:47:35.513 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:47:35.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:35.514 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:35.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:47:35.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:47:35.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:47:35.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:117: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T06:47:35.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:117: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-17T06:47:35.763 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:117: TEST_scrub_test: test 2 = 2 2026-02-17T06:47:35.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:118: TEST_scrub_test: ceph pg dump pgs 2026-02-17T06:47:35.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:118: TEST_scrub_test: grep '^1.0' 2026-02-17T06:47:35.764 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:118: TEST_scrub_test: grep -vq -- +inconsistent 2026-02-17T06:47:35.901 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:47:35.910 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:120: TEST_scrub_test: ceph osd out 1 2026-02-17T06:47:36.104 INFO:tasks.workunit.client.0.trial074.stderr:marked out osd.1. 2026-02-17T06:47:36.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:121: TEST_scrub_test: wait_for_clean 2026-02-17T06:47:36.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:47:36.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:47:36.115 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:47:36.115 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:47:36.116 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:47:36.117 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:47:36.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:47:36.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:47:36.117 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:47:36.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:47:36.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:47:36.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:47:36.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:47:36.234 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T06:47:36.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:47:36.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:47:36.385 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:47:36.386 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:47:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:47:36.386 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:36.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:47:36.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836545 2026-02-17T06:47:36.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836545 2026-02-17T06:47:36.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545' 2026-02-17T06:47:36.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:36.455 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:47:36.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705720 2026-02-17T06:47:36.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705720 2026-02-17T06:47:36.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545 1-38654705720' 2026-02-17T06:47:36.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:47:36.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:47:36.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=94489280548 2026-02-17T06:47:36.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 94489280548 2026-02-17T06:47:36.597 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545 1-38654705720 2-94489280548' 2026-02-17T06:47:36.598 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:36.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836545 2026-02-17T06:47:36.598 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:36.599 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:47:36.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836545 2026-02-17T06:47:36.600 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:36.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836545 2026-02-17T06:47:36.601 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836545 2026-02-17T06:47:36.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836545' 2026-02-17T06:47:36.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:36.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836543 -lt 21474836545 2026-02-17T06:47:36.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:47:37.762 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T06:47:37.763 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:47:37.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836546 -lt 21474836545 2026-02-17T06:47:37.915 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:37.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705720 2026-02-17T06:47:37.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:37.916 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:47:37.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705720 2026-02-17T06:47:37.917 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:37.919 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705720 2026-02-17T06:47:37.919 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 38654705720 2026-02-17T06:47:37.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705720' 2026-02-17T06:47:37.920 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:47:38.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705721 -lt 38654705720 2026-02-17T06:47:38.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:47:38.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-94489280548 2026-02-17T06:47:38.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:47:38.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:47:38.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-94489280548 2026-02-17T06:47:38.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:47:38.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=94489280548 2026-02-17T06:47:38.078 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 94489280548 2026-02-17T06:47:38.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 94489280548' 2026-02-17T06:47:38.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:47:38.235 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 94489280549 -lt 94489280548 2026-02-17T06:47:38.236 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:47:38.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:38.237 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:47:38.419 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:47:38.569 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:47:38.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:47:38.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:47:38.570 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:47:38.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:47:38.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:47:38.761 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:47:38.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T06:47:38.762 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-17T06:47:38.830 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: test 0 = 0 2026-02-17T06:47:38.831 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:124: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T06:47:38.831 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:124: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-17T06:47:38.904 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:124: TEST_scrub_test: test 0 = 0 2026-02-17T06:47:38.904 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:125: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T06:47:38.905 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:125: TEST_scrub_test: jq '.peer_info[1].stats.stat_sum.num_scrub_errors' 2026-02-17T06:47:38.974 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:125: TEST_scrub_test: test 0 = 0 2026-02-17T06:47:38.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:126: TEST_scrub_test: ceph pg dump pgs 2026-02-17T06:47:38.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:126: TEST_scrub_test: grep '^1.0' 2026-02-17T06:47:38.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:126: TEST_scrub_test: grep -vq -- +inconsistent 2026-02-17T06:47:39.113 INFO:tasks.workunit.client.0.trial074.stderr:dumped pgs 2026-02-17T06:47:39.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:127: TEST_scrub_test: perf_counters td/osd-scrub-test 3 2026-02-17T06:47:39.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T06:47:39.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T06:47:39.123 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T06:47:39.124 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T06:47:39.125 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:47:39.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T06:47:39.126 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.197 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.198 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.199 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.200 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 1, 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 1, 2026-02-17T06:47:39.201 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 1, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0.003999948, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0.003999948 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.202 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 2, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 1, 2026-02-17T06:47:39.203 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.204 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.205 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.206 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.207 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.208 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.209 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.210 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.210 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.210 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.210 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.210 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.210 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.211 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.212 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.212 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.212 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.212 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:47:39.212 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:47:39.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:47:39.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T06:47:39.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:47:39.265 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.266 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.267 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.268 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.269 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 3, 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 3, 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 3, 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 3, 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0.01199984, 2026-02-17T06:47:39.270 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0.003999946 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 6, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.271 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 3, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.272 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 2 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.273 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.274 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.275 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.276 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.277 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.278 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.279 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.279 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.279 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.279 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.279 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.279 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:47:39.280 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:47:39.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T06:47:39.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T06:47:39.281 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T06:47:39.337 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:47:39.337 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_ec": [ 2026-02-17T06:47:39.337 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.337 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.337 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.338 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.339 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.340 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_dp_repl": [ 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.341 INFO:tasks.workunit.client.0.trial074.stdout: "level": "deep", 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.342 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.343 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.344 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_ec": [ 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "ec" 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.345 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.346 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.347 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: ], 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "osd_scrub_sh_repl": [ 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: { 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "labels": { 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "level": "shallow", 2026-02-17T06:47:39.348 INFO:tasks.workunit.client.0.trial074.stdout: "pooltype": "replicated" 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "counters": { 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_started": 0, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs": 0, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "successful_scrubs_elapsed": { 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs": 0, 2026-02-17T06:47:39.349 INFO:tasks.workunit.client.0.trial074.stdout: "failed_scrubs_elapsed": { 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "preemptions": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_selected": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "chunk_busy": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "locked_object": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "write_blocked_by_scrub": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "scrub_reservations_completed": 0, 2026-02-17T06:47:39.350 INFO:tasks.workunit.client.0.trial074.stdout: "successful_reservations_elapsed": { 2026-02-17T06:47:39.351 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.351 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.351 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.351 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.351 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_aborted": 0, 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_failure": 0, 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: "reservation_process_skipped": 0, 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: "failed_reservations_elapsed": { 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgcount": 0, 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: "sum": 0, 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: "avgtime": 0 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: }, 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: "replicas_in_reservation": 0 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: } 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout: ] 2026-02-17T06:47:39.352 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:47:39.353 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Teardown Test TEST_scrub_test ------------------ 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_test ------------------' 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:47:39.354 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:47:39.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:47:39.355 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:47:39.655 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:47:39.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:47:39.657 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:47:39.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:47:39.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:47:39.659 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:47:39.660 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:47:39.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:47:39.661 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:47:39.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:47:39.662 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:47:39.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:47:39.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:47:39.664 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:47:39.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:47:39.678 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:39.679 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:47:39.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:47:39.679 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:47:39.680 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:47:39.680 INFO:tasks.workunit.client.0.trial074.stdout:-------------- Complete Test TEST_scrub_test ------------------ 2026-02-17T06:47:39.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_test ------------------' 2026-02-17T06:47:39.681 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-test 0 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:47:39.682 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:47:39.683 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:47:39.683 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:47:39.684 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:47:39.684 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:47:39.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:47:39.685 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:47:39.686 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:47:39.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:47:39.686 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:47:39.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:47:39.687 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:47:39.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:47:39.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T06:47:39.689 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T06:47:39.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:47:39.690 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:39.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.507902 2026-02-17T06:47:39.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.507902 2026-02-17T06:47:39.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:47:39.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:47:39.692 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T06:47:39.693 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T06:47:39.693 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T06:47:39.743 INFO:tasks.workunit:Running workunit scrub/osd-unexpected-clone.sh... 2026-02-17T06:47:39.744 DEBUG:teuthology.orchestra.run.trial074:workunit test scrub/osd-unexpected-clone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=04cf6dd4be446d06026adad18a5fbf5da1fa8c0e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh 2026-02-17T06:47:39.796 INFO:tasks.workunit.client.0.trial074.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T06:47:39.800 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-unexpected-clone 2026-02-17T06:47:39.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:22: run: local dir=td/osd-unexpected-clone 2026-02-17T06:47:39.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:23: run: shift 2026-02-17T06:47:39.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:25: run: export CEPH_MON=127.0.0.1:7144 2026-02-17T06:47:39.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:25: run: CEPH_MON=127.0.0.1:7144 2026-02-17T06:47:39.801 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:26: run: export CEPH_ARGS 2026-02-17T06:47:39.801 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:27: run: uuidgen 2026-02-17T06:47:39.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:27: run: CEPH_ARGS+='--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none ' 2026-02-17T06:47:39.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7144 ' 2026-02-17T06:47:39.802 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:30: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T06:47:39.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: set 2026-02-17T06:47:39.802 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: local funcs=TEST_recover_unexpected 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:32: run: for func in $funcs 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:33: run: setup td/osd-unexpected-clone 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-unexpected-clone 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-unexpected-clone 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-unexpected-clone 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:47:39.804 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-17T06:47:39.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:47:39.805 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:47:39.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:47:39.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:47:39.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:47:39.805 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:47:39.806 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:47:39.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:47:39.807 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:47:39.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:47:39.808 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:47:39.809 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:47:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:47:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:47:39.810 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:47:39.811 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:47:39.811 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:47:39.812 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:47:39.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-unexpected-clone 2026-02-17T06:47:39.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:47:39.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:39.814 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:39.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.648547 2026-02-17T06:47:39.815 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:47:39.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:47:39.816 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-unexpected-clone 2026-02-17T06:47:39.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T06:47:39.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:39.817 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:39.818 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.648547 2026-02-17T06:47:39.819 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T06:47:39.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T06:47:39.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-unexpected-clone 1' TERM HUP INT 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:34: run: TEST_recover_unexpected td/osd-unexpected-clone 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:40: TEST_recover_unexpected: local dir=td/osd-unexpected-clone 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:42: TEST_recover_unexpected: run_mon td/osd-unexpected-clone a 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-unexpected-clone 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-unexpected-clone/a 2026-02-17T06:47:39.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-unexpected-clone/a --run-dir=td/osd-unexpected-clone 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:39.844 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-unexpected-clone/a '--log-file=td/osd-unexpected-clone/$name.log' '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --mon-cluster-log-file=td/osd-unexpected-clone/log --run-dir=td/osd-unexpected-clone '--pid-file=td/osd-unexpected-clone/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T06:47:39.871 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T06:47:39.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T06:47:39.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:47:39.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:47:39.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T06:47:39.872 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T06:47:39.872 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:47:39.873 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:47:39.873 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:47:39.874 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:47:39.874 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:39.874 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:39.874 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.648547/ceph-mon.a.asok 2026-02-17T06:47:39.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:47:39.874 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.648547/ceph-mon.a.asok config get fsid 2026-02-17T06:47:39.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T06:47:39.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T06:47:39.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.648547/ceph-mon.a.asok 2026-02-17T06:47:39.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T06:47:39.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.648547/ceph-mon.a.asok config get mon_host 2026-02-17T06:47:39.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:43: TEST_recover_unexpected: run_mgr td/osd-unexpected-clone x 2026-02-17T06:47:39.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-unexpected-clone 2026-02-17T06:47:39.984 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T06:47:39.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T06:47:39.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T06:47:39.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-unexpected-clone/x 2026-02-17T06:47:39.985 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T06:47:40.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T06:47:40.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:47:40.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:47:40.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:47:40.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:40.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:40.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:40.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:47:40.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-unexpected-clone/x '--log-file=td/osd-unexpected-clone/$name.log' '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --run-dir=td/osd-unexpected-clone '--pid-file=td/osd-unexpected-clone/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T06:47:40.100 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:44: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 0 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-unexpected-clone/0 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:47:40.101 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/0' 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/0/journal' 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:40.102 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:47:40.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:47:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:47:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-unexpected-clone/0 2026-02-17T06:47:40.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:47:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=80ee725a-155e-43cb-9b8d-cfec4d96a71f 2026-02-17T06:47:40.104 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 80ee725a-155e-43cb-9b8d-cfec4d96a71f' 2026-02-17T06:47:40.104 INFO:tasks.workunit.client.0.trial074.stdout:add osd0 80ee725a-155e-43cb-9b8d-cfec4d96a71f 2026-02-17T06:47:40.105 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:47:40.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAMD5RpLvYmBhAAYsSHzbZn+rgPGwKQtSyqWg== 2026-02-17T06:47:40.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAMD5RpLvYmBhAAYsSHzbZn+rgPGwKQtSyqWg=="}' 2026-02-17T06:47:40.110 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 80ee725a-155e-43cb-9b8d-cfec4d96a71f -i td/osd-unexpected-clone/0/new.json 2026-02-17T06:47:40.196 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:47:40.204 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-unexpected-clone/0/new.json 2026-02-17T06:47:40.205 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/0 --osd-journal=td/osd-unexpected-clone/0/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMD5RpLvYmBhAAYsSHzbZn+rgPGwKQtSyqWg== --osd-uuid 80ee725a-155e-43cb-9b8d-cfec4d96a71f 2026-02-17T06:47:40.220 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:40.210+0000 7f043efb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:40.221 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:40.210+0000 7f043efb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:40.223 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:40.210+0000 7f043efb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:40.223 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:40.210+0000 7f043efb28c0 -1 bdev(0x561caa2c3c00 td/osd-unexpected-clone/0/block) open stat got: (1) Operation not permitted 2026-02-17T06:47:40.223 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:40.210+0000 7f043efb28c0 -1 bluestore(td/osd-unexpected-clone/0) _read_fsid unparsable uuid 2026-02-17T06:47:43.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-unexpected-clone/0/keyring 2026-02-17T06:47:43.076 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:47:43.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T06:47:43.077 INFO:tasks.workunit.client.0.trial074.stdout:adding osd0 key to auth repository 2026-02-17T06:47:43.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-unexpected-clone/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:47:43.275 INFO:tasks.workunit.client.0.trial074.stdout:start osd.0 2026-02-17T06:47:43.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T06:47:43.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/0 --osd-journal=td/osd-unexpected-clone/0/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:47:43.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:47:43.275 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:47:43.276 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:47:43.297 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:43.286+0000 7fc63245e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:43.299 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:43.290+0000 7fc63245e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:43.300 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:43.290+0000 7fc63245e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T06:47:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:47:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T06:47:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:47:43.431 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:47:43.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:43.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:47:43.432 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:47:43.432 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:43.433 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:47:43.582 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:44.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:44.583 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:44.584 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:47:44.584 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:47:44.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:44.585 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:47:44.730 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:44.772 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:44.762+0000 7fc63245e8c0 -1 Falling back to public interface 2026-02-17T06:47:45.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:45.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:45.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:47:45.731 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:47:45.731 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:45.732 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:47:45.882 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:46.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:46.884 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:46.884 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:47:46.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:47:46.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:46.885 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:47:47.034 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:47.739 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:47.730+0000 7fc63245e8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T06:47:48.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:48.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:48.036 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:47:48.036 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:47:48.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:48.037 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:47:48.197 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:49.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:49.198 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:49.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:47:49.199 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:47:49.199 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:49.200 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2093138098,v1:127.0.0.1:6803/2093138098] [v2:127.0.0.1:6804/2093138098,v1:127.0.0.1:6805/2093138098] exists,up 80ee725a-155e-43cb-9b8d-cfec4d96a71f 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:45: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 1 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:47:49.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:49.350 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:47:49.351 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-17T06:47:49.352 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:47:49.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=54114ee9-b8ed-48af-9c1f-5994ee94bf1d 2026-02-17T06:47:49.353 INFO:tasks.workunit.client.0.trial074.stdout:add osd1 54114ee9-b8ed-48af-9c1f-5994ee94bf1d 2026-02-17T06:47:49.353 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 54114ee9-b8ed-48af-9c1f-5994ee94bf1d' 2026-02-17T06:47:49.353 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:47:49.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAVD5RpsrBZFRAAg5DuWhLE4lX3uXZ/gmv6UQ== 2026-02-17T06:47:49.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAVD5RpsrBZFRAAg5DuWhLE4lX3uXZ/gmv6UQ=="}' 2026-02-17T06:47:49.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 54114ee9-b8ed-48af-9c1f-5994ee94bf1d -i td/osd-unexpected-clone/1/new.json 2026-02-17T06:47:49.522 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:47:49.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-unexpected-clone/1/new.json 2026-02-17T06:47:49.532 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAVD5RpsrBZFRAAg5DuWhLE4lX3uXZ/gmv6UQ== --osd-uuid 54114ee9-b8ed-48af-9c1f-5994ee94bf1d 2026-02-17T06:47:49.547 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:49.538+0000 7f39bcb2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:49.548 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:49.538+0000 7f39bcb2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:49.549 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:49.538+0000 7f39bcb2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:49.550 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:49.538+0000 7f39bcb2e8c0 -1 bdev(0x564632bb7c00 td/osd-unexpected-clone/1/block) open stat got: (1) Operation not permitted 2026-02-17T06:47:49.550 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:49.538+0000 7f39bcb2e8c0 -1 bluestore(td/osd-unexpected-clone/1) _read_fsid unparsable uuid 2026-02-17T06:47:51.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-unexpected-clone/1/keyring 2026-02-17T06:47:51.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:47:51.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T06:47:51.814 INFO:tasks.workunit.client.0.trial074.stdout:adding osd1 key to auth repository 2026-02-17T06:47:51.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-unexpected-clone/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:47:52.012 INFO:tasks.workunit.client.0.trial074.stdout:start osd.1 2026-02-17T06:47:52.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T06:47:52.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:47:52.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:47:52.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:47:52.013 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:47:52.033 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:52.022+0000 7f785c2518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:52.034 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:52.022+0000 7f785c2518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:52.035 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:52.026+0000 7f785c2518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:52.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T06:47:52.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:47:52.165 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:47:52.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:47:52.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:47:52.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:52.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:47:52.166 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:47:52.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:52.167 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:47:52.319 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:53.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:53.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:53.321 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:47:53.321 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:47:53.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:53.322 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:47:53.471 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:53.540 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:53.530+0000 7f785c2518c0 -1 Falling back to public interface 2026-02-17T06:47:54.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:54.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:54.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:47:54.473 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:47:54.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:54.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:47:54.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:55.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:55.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:55.624 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:47:55.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:47:55.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:55.624 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:47:55.773 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:56.531 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:56.518+0000 7f785c2518c0 -1 osd.1 0 log_to_monitors true 2026-02-17T06:47:56.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:56.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:56.775 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:47:56.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:47:56.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:56.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:47:56.925 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:47:57.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:47:57.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:47:57.928 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:47:57.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:47:57.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:47:57.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:47:58.079 INFO:tasks.workunit.client.0.trial074.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/818258456,v1:127.0.0.1:6811/818258456] [v2:127.0.0.1:6812/818258456,v1:127.0.0.1:6813/818258456] exists,up 54114ee9-b8ed-48af-9c1f-5994ee94bf1d 2026-02-17T06:47:58.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:47:58.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:47:58.082 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:46: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 2 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:47:58.083 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:47:58.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:47:58.085 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:47:58.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:47:58.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:47:58.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T06:47:58.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T06:47:58.086 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T06:47:58.087 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T06:47:58.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5eab9662-6bfd-45c5-a1e3-4a9c644734bd 2026-02-17T06:47:58.087 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 5eab9662-6bfd-45c5-a1e3-4a9c644734bd' 2026-02-17T06:47:58.087 INFO:tasks.workunit.client.0.trial074.stdout:add osd2 5eab9662-6bfd-45c5-a1e3-4a9c644734bd 2026-02-17T06:47:58.088 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T06:47:58.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAeD5RpewVMBRAA9MDh+ZxocSnSB4df/yHtsA== 2026-02-17T06:47:58.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAeD5RpewVMBRAA9MDh+ZxocSnSB4df/yHtsA=="}' 2026-02-17T06:47:58.096 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5eab9662-6bfd-45c5-a1e3-4a9c644734bd -i td/osd-unexpected-clone/2/new.json 2026-02-17T06:47:58.240 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:47:58.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-unexpected-clone/2/new.json 2026-02-17T06:47:58.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAeD5RpewVMBRAA9MDh+ZxocSnSB4df/yHtsA== --osd-uuid 5eab9662-6bfd-45c5-a1e3-4a9c644734bd 2026-02-17T06:47:58.267 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:58.258+0000 7f5c781718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:58.268 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:58.258+0000 7f5c781718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:58.269 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:58.258+0000 7f5c781718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:47:58.270 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:58.258+0000 7f5c781718c0 -1 bdev(0x55792fe53c00 td/osd-unexpected-clone/2/block) open stat got: (1) Operation not permitted 2026-02-17T06:47:58.270 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:47:58.258+0000 7f5c781718c0 -1 bluestore(td/osd-unexpected-clone/2) _read_fsid unparsable uuid 2026-02-17T06:48:00.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-unexpected-clone/2/keyring 2026-02-17T06:48:00.564 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T06:48:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T06:48:00.565 INFO:tasks.workunit.client.0.trial074.stdout:adding osd2 key to auth repository 2026-02-17T06:48:00.565 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-unexpected-clone/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T06:48:00.757 INFO:tasks.workunit.client.0.trial074.stdout:start osd.2 2026-02-17T06:48:00.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T06:48:00.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:48:00.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T06:48:00.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T06:48:00.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T06:48:00.779 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:00.766+0000 7fe006f058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:00.780 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:00.770+0000 7fe006f058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:00.782 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:00.770+0000 7fe006f058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:00.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T06:48:00.907 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:48:00.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:48:00.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:48:00.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:48:00.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:00.908 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:48:00.908 INFO:tasks.workunit.client.0.trial074.stdout:0 2026-02-17T06:48:00.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:00.909 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:01.056 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:02.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:02.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:02.058 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:48:02.059 INFO:tasks.workunit.client.0.trial074.stdout:1 2026-02-17T06:48:02.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:02.059 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:02.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:02.252 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:02.242+0000 7fe006f058c0 -1 Falling back to public interface 2026-02-17T06:48:03.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:03.211 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:03.212 INFO:tasks.workunit.client.0.trial074.stdout:2 2026-02-17T06:48:03.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:48:03.212 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:03.213 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:03.362 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:04.363 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:04.364 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:04.364 INFO:tasks.workunit.client.0.trial074.stdout:3 2026-02-17T06:48:04.365 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:48:04.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:04.366 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:04.512 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:05.269 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:05.258+0000 7fe006f058c0 -1 osd.2 0 log_to_monitors true 2026-02-17T06:48:05.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:05.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:05.514 INFO:tasks.workunit.client.0.trial074.stdout:4 2026-02-17T06:48:05.514 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:48:05.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:05.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:05.660 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:06.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:06.662 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:06.662 INFO:tasks.workunit.client.0.trial074.stdout:5 2026-02-17T06:48:06.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T06:48:06.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:06.663 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:06.813 INFO:tasks.workunit.client.0.trial074.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2024530616,v1:127.0.0.1:6819/2024530616] [v2:127.0.0.1:6820/2024530616,v1:127.0.0.1:6821/2024530616] exists,up 5eab9662-6bfd-45c5-a1e3-4a9c644734bd 2026-02-17T06:48:06.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:48:06.813 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:48:06.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:48:06.814 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:48: TEST_recover_unexpected: ceph osd pool create foo 1 2026-02-17T06:48:07.009 INFO:tasks.workunit.client.0.trial074.stderr:pool 'foo' created 2026-02-17T06:48:07.019 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:49: TEST_recover_unexpected: rados -p foo put foo /etc/passwd 2026-02-17T06:48:08.240 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:50: TEST_recover_unexpected: rados -p foo mksnap snap 2026-02-17T06:48:08.286 INFO:tasks.workunit.client.0.trial074.stdout:created pool foo snap snap 2026-02-17T06:48:08.288 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:51: TEST_recover_unexpected: rados -p foo put foo /etc/group 2026-02-17T06:48:08.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:53: TEST_recover_unexpected: wait_for_clean 2026-02-17T06:48:08.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:48:08.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:48:08.312 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:48:08.312 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:48:08.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:48:08.313 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:48:08.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:48:08.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:48:08.314 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:48:08.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:48:08.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:48:08.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:48:08.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:48:08.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:48:08.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:48:08.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:48:08.574 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:08.574 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:48:08.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:48:08.574 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:08.575 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:48:08.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T06:48:08.644 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T06:48:08.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T06:48:08.645 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:08.645 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:48:08.716 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T06:48:08.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T06:48:08.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T06:48:08.717 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:08.717 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:48:08.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509443 2026-02-17T06:48:08.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509443 2026-02-17T06:48:08.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509443' 2026-02-17T06:48:08.789 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:08.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T06:48:08.790 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:08.791 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:48:08.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T06:48:08.792 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:08.793 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T06:48:08.793 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.0 seq 21474836485 2026-02-17T06:48:08.794 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T06:48:08.794 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:08.948 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:48:08.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:48:09.949 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:48:09.950 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:10.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T06:48:10.102 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:48:11.103 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:48:11.104 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:11.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T06:48:11.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:11.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T06:48:11.256 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:11.257 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:48:11.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T06:48:11.258 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:11.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T06:48:11.259 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T06:48:11.260 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.1 seq 42949672964 2026-02-17T06:48:11.260 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:48:11.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T06:48:11.412 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:11.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509443 2026-02-17T06:48:11.413 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:11.414 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:48:11.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509443 2026-02-17T06:48:11.415 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:11.416 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509443 2026-02-17T06:48:11.417 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-17T06:48:11.417 INFO:tasks.workunit.client.0.trial074.stdout:waiting osd.2 seq 64424509443 2026-02-17T06:48:11.417 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:48:11.568 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-17T06:48:11.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:48:11.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:11.569 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:11.747 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:48:11.748 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:11.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:11.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:11.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:11.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:11.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:11.749 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:11.895 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:48:11.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:11.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:11.896 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:48:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:48:12.075 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:48:12.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:55: TEST_recover_unexpected: get_primary foo foo 2026-02-17T06:48:12.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=foo 2026-02-17T06:48:12.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=foo 2026-02-17T06:48:12.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map foo foo 2026-02-17T06:48:12.076 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T06:48:12.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:55: TEST_recover_unexpected: local osd=1 2026-02-17T06:48:12.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 1 --op list foo 2026-02-17T06:48:12.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: grep snapid.:1 2026-02-17T06:48:12.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T06:48:12.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 1 --op list foo 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.1 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:48:12.229 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:48:12.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:48:13.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:48:13.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 1 --op list foo 2026-02-17T06:48:13.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T06:48:13.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:48:13.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:48:13.537 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:48:13.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/1 2026-02-17T06:48:13.538 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/1 --op list foo 2026-02-17T06:48:14.383 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 1 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:48:14.384 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:48:14.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-17T06:48:14.386 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:48:14.387 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:48:14.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:48:14.388 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/1/whoami 2026-02-17T06:48:14.388 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:48:14.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:48:14.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:48:14.389 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:48:14.404 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:14.394+0000 7f96e2aeb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:14.406 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:14.394+0000 7f96e2aeb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:14.407 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:14.398+0000 7f96e2aeb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:14.541 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:48:14.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:48:14.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:48:14.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:48:14.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:48:14.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:14.542 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:48:14.542 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:48:14.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:14.543 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:14.691 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:15.637 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:15.626+0000 7f96e2aeb8c0 -1 Falling back to public interface 2026-02-17T06:48:15.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:15.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:15.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:48:15.693 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:15.693 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:15.694 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:15.845 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:16.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:16.847 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:16.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:48:16.848 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:48:16.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:16.848 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:16.999 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:18.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:18.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:18.001 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:48:18.001 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:48:18.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:18.002 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:18.151 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:18.654 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:18.642+0000 7f96e2aeb8c0 -1 osd.1 19 log_to_monitors true 2026-02-17T06:48:19.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:19.152 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:19.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:48:19.153 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:48:19.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:19.153 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:19.299 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 23 up_thru 23 down_at 20 last_clean_interval [10,19) [v2:127.0.0.1:6810/781192100,v1:127.0.0.1:6811/781192100] [v2:127.0.0.1:6812/781192100,v1:127.0.0.1:6813/781192100] exists,up 54114ee9-b8ed-48af-9c1f-5994ee94bf1d 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:48:19.300 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:48:19.301 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:48:19.301 INFO:tasks.workunit.client.0.trial074.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:48:19.301 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:48:19.301 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:48:19.302 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:48:19.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:48:19.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:48:19.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:48:19.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:48:19.397 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:48:19.397 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:48:19.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:48:19.553 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:19.553 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:48:19.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:48:19.553 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:19.553 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:48:19.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T06:48:19.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T06:48:19.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T06:48:19.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:19.624 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:48:19.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T06:48:19.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T06:48:19.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-98784247810' 2026-02-17T06:48:19.697 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:19.697 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:48:19.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509446 2026-02-17T06:48:19.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509446 2026-02-17T06:48:19.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-98784247810 2-64424509446' 2026-02-17T06:48:19.765 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:19.765 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T06:48:19.766 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:19.766 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:48:19.767 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T06:48:19.767 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:19.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T06:48:19.769 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T06:48:19.769 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836489 2026-02-17T06:48:19.769 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:19.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-17T06:48:19.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:48:20.928 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:48:20.928 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:21.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T06:48:21.078 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:21.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-98784247810 2026-02-17T06:48:21.079 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:21.080 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:48:21.080 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-98784247810 2026-02-17T06:48:21.080 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:21.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T06:48:21.082 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 98784247810' 2026-02-17T06:48:21.082 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 98784247810 2026-02-17T06:48:21.082 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:48:21.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T06:48:21.231 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:21.232 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509446 2026-02-17T06:48:21.232 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:21.232 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:48:21.233 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509446 2026-02-17T06:48:21.233 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:21.234 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509446 2026-02-17T06:48:21.235 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-17T06:48:21.235 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509446 2026-02-17T06:48:21.235 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:48:21.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-17T06:48:21.402 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:48:21.402 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:21.402 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:21.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:48:21.582 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:21.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:21.582 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:21.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:21.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:21.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:21.583 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:21.732 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:48:21.732 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:21.733 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:21.733 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:21.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:48:21.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:48:21.916 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:48:21.917 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: JSON='["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:48:21.917 INFO:tasks.workunit.client.0.trial074.stdout:JSON is ["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}] 2026-02-17T06:48:21.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:58: TEST_recover_unexpected: echo 'JSON is ["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T06:48:21.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:59: TEST_recover_unexpected: rm -f td/osd-unexpected-clone/_ td/osd-unexpected-clone/data 2026-02-17T06:48:21.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:60: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T06:48:21.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T06:48:21.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:48:21.920 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.1 2026-02-17T06:48:21.921 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:48:21.922 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:48:21.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:48:21.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:48:21.922 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:48:23.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:48:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T06:48:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T06:48:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:48:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T06:48:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:48:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/1 2026-02-17T06:48:23.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T06:48:24.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 1 2026-02-17T06:48:24.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:48:24.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:48:24.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:48:24.074 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:48:24.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:48:24.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:48:24.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:48:24.075 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:48:24.075 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:48:24.075 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:48:24.075 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:24.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:24.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:48:24.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:48:24.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:48:24.077 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:start osd.1 2026-02-17T06:48:24.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:48:24.079 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/1/whoami 2026-02-17T06:48:24.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T06:48:24.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:48:24.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:48:24.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:48:24.095 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:24.086+0000 7f1f580cb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:24.097 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:24.086+0000 7f1f580cb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:24.098 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:24.086+0000 7f1f580cb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:24.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:24.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:24.383 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:25.373 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:25.362+0000 7f1f580cb8c0 -1 Falling back to public interface 2026-02-17T06:48:25.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:25.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:25.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:48:25.384 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:25.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:25.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:25.537 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:26.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:26.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:26.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:48:26.539 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:48:26.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:26.539 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:26.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:27.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:27.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:27.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:48:27.690 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:48:27.690 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:27.691 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:27.840 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:28.356 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:28.346+0000 7f1f580cb8c0 -1 osd.1 24 log_to_monitors true 2026-02-17T06:48:28.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:28.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:28.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:48:28.842 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:48:28.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:28.842 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T06:48:28.992 INFO:tasks.workunit.client.0.trial074.stderr:osd.1 up in weight 1 up_from 28 up_thru 28 down_at 25 last_clean_interval [23,24) [v2:127.0.0.1:6810/293972141,v1:127.0.0.1:6811/293972141] [v2:127.0.0.1:6812/293972141,v1:127.0.0.1:6813/293972141] exists,up 54114ee9-b8ed-48af-9c1f-5994ee94bf1d 2026-02-17T06:48:28.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:48:28.993 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:48:28.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:48:28.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:48:28.994 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:48:29.093 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:48:29.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:48:29.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:48:29.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:48:29.094 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:48:29.094 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:48:29.248 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:48:29.249 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:29.249 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:48:29.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:48:29.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:29.249 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:48:29.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T06:48:29.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T06:48:29.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T06:48:29.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:29.320 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:48:29.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084290 2026-02-17T06:48:29.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084290 2026-02-17T06:48:29.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-120259084290' 2026-02-17T06:48:29.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:29.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:48:29.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509449 2026-02-17T06:48:29.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509449 2026-02-17T06:48:29.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-120259084290 2-64424509449' 2026-02-17T06:48:29.459 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:29.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T06:48:29.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:29.460 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:48:29.460 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T06:48:29.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:29.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T06:48:29.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T06:48:29.462 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836492 2026-02-17T06:48:29.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:29.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-17T06:48:29.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:48:30.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:48:30.623 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:30.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-17T06:48:30.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:30.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-120259084290 2026-02-17T06:48:30.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:30.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:48:30.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-120259084290 2026-02-17T06:48:30.774 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084290 2026-02-17T06:48:30.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 120259084290' 2026-02-17T06:48:30.776 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 120259084290 2026-02-17T06:48:30.776 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:48:30.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084290 -lt 120259084290 2026-02-17T06:48:30.927 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:30.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509449 2026-02-17T06:48:30.927 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:30.928 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:48:30.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509449 2026-02-17T06:48:30.929 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:30.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509449 2026-02-17T06:48:30.931 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-17T06:48:30.931 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 64424509449 2026-02-17T06:48:30.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:48:31.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-17T06:48:31.084 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:48:31.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:31.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:31.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:48:31.269 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:31.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:31.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:31.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:31.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:31.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:31.270 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:31.421 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:48:31.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:31.421 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:31.422 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:31.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:61: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T06:48:31.605 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:48:31.606 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:48:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:48:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T06:48:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:48:32.912 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:48:32.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:48:32.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-17T06:48:32.913 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:48:33.772 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:48:33.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:48:33.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:48:33.773 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:48:33.773 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:48:33.773 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:48:33.773 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:48:33.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T06:48:33.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:48:33.775 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:48:33.775 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:48:33.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-17T06:48:33.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:48:33.776 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:48:33.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:48:33.777 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:48:33.793 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:33.782+0000 7eff9f0798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:33.794 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:33.782+0000 7eff9f0798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:33.795 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:33.786+0000 7eff9f0798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:48:33.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:33.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:34.078 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:35.045 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:35.034+0000 7eff9f0798c0 -1 Falling back to public interface 2026-02-17T06:48:35.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:35.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:35.079 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:48:35.080 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:35.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:35.080 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:35.227 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:36.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:36.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:36.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:48:36.229 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:48:36.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:36.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:36.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:37.388 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:48:37.389 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:48:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:37.389 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:37.522 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:38.064 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:38.054+0000 7eff9f0798c0 -1 osd.2 29 log_to_monitors true 2026-02-17T06:48:38.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:38.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:38.523 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:48:38.523 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:48:38.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:38.524 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:38.673 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 33 up_thru 0 down_at 30 last_clean_interval [15,29) [v2:127.0.0.1:6818/3013721485,v1:127.0.0.1:6819/3013721485] [v2:127.0.0.1:6820/3013721485,v1:127.0.0.1:6821/3013721485] exists,up 5eab9662-6bfd-45c5-a1e3-4a9c644734bd 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:48:38.674 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:48:38.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:48:38.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:48:38.675 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:48:38.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:48:38.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:48:38.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:48:38.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:48:38.774 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:48:38.775 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:48:38.929 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:48:38.930 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:38.930 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:48:38.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:48:38.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:38.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:48:38.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836495 2026-02-17T06:48:38.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836495 2026-02-17T06:48:38.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495' 2026-02-17T06:48:38.999 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:39.000 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:48:39.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084293 2026-02-17T06:48:39.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084293 2026-02-17T06:48:39.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-120259084293' 2026-02-17T06:48:39.071 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:39.072 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:48:39.140 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920770 2026-02-17T06:48:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920770 2026-02-17T06:48:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-120259084293 2-141733920770' 2026-02-17T06:48:39.141 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:39.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836495 2026-02-17T06:48:39.141 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:39.142 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:48:39.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836495 2026-02-17T06:48:39.143 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:39.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836495 2026-02-17T06:48:39.144 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-17T06:48:39.144 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836495 2026-02-17T06:48:39.144 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:39.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-17T06:48:39.304 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:48:40.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:48:40.305 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:40.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-17T06:48:40.457 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:40.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-120259084293 2026-02-17T06:48:40.457 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:40.458 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:48:40.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-120259084293 2026-02-17T06:48:40.459 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:40.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084293 2026-02-17T06:48:40.461 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 120259084293' 2026-02-17T06:48:40.461 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 120259084293 2026-02-17T06:48:40.461 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:48:40.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084293 -lt 120259084293 2026-02-17T06:48:40.622 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:40.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-141733920770 2026-02-17T06:48:40.622 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:40.623 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:48:40.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-141733920770 2026-02-17T06:48:40.624 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:40.625 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920770 2026-02-17T06:48:40.626 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 141733920770' 2026-02-17T06:48:40.626 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 141733920770 2026-02-17T06:48:40.626 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:48:40.784 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920770 2026-02-17T06:48:40.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:48:40.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:40.785 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:40.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:48:40.971 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:40.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:40.971 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:40.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:40.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:40.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:40.972 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:41.120 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:48:41.120 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:41.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:41.121 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:41.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:48:41.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:48:41.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:48:41.306 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:63: TEST_recover_unexpected: rados -p foo rmsnap snap 2026-02-17T06:48:41.372 INFO:tasks.workunit.client.0.trial074.stdout:removed pool foo snap snap 2026-02-17T06:48:41.374 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:65: TEST_recover_unexpected: sleep 5 2026-02-17T06:48:46.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:67: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:46.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T06:48:46.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:48:46.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:48:46.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:48:46.375 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:46.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T06:48:46.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:48:46.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:48:46.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:48:46.376 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-17T06:48:46.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:48:46.377 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:48:46.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:48:46.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:48:46.377 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:48:47.687 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:48:47.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:47.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T06:48:47.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:48:47.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:48:47.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:48:47.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-17T06:48:47.688 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T06:48:48.857 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-17T06:48:48.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:48:48.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:48:48.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:48:48.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:48:48.858 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T06:48:48.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:48:48.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:48:48.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:48:48.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:48:48.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T06:48:48.859 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T06:48:48.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:48:48.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:48:48.860 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:48:48.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:48:48.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:48:48.860 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:48:48.860 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:48:48.861 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:48:48.861 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:48:48.861 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:48:48.863 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:48:48.864 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:48:48.877 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:48.865+0000 7f98ec48e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:48.878 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:48.865+0000 7f98ec48e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:48.879 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:48.869+0000 7f98ec48e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:49.011 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:49.164 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:50.128 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:50.117+0000 7f98ec48e8c0 -1 Falling back to public interface 2026-02-17T06:48:50.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:50.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:50.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:48:50.166 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:50.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:50.166 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:50.318 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:51.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:51.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:51.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:48:51.320 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:48:51.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:51.320 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:51.472 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:52.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:52.473 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:52.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:48:52.474 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:48:52.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:52.474 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:52.617 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:48:53.125 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:48:53.113+0000 7f98ec48e8c0 -1 osd.2 35 log_to_monitors true 2026-02-17T06:48:53.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:48:53.619 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:48:53.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:48:53.620 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:48:53.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:48:53.620 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 40 up_thru 0 down_at 37 last_clean_interval [33,35) [v2:127.0.0.1:6818/634783920,v1:127.0.0.1:6819/634783920] [v2:127.0.0.1:6820/634783920,v1:127.0.0.1:6821/634783920] exists,up 5eab9662-6bfd-45c5-a1e3-4a9c644734bd 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:48:53.767 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:48:53.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:48:53.768 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:48:53.768 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:48:53.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:48:53.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:48:53.768 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:48:53.869 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:48:53.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:48:53.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:48:53.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:48:53.870 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:48:53.870 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:48:54.028 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:48:54.029 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:48:54.029 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:48:54.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:48:54.029 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:54.029 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:48:54.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836499 2026-02-17T06:48:54.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836499 2026-02-17T06:48:54.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499' 2026-02-17T06:48:54.099 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:54.100 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:48:54.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084297 2026-02-17T06:48:54.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084297 2026-02-17T06:48:54.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-120259084297' 2026-02-17T06:48:54.171 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:48:54.171 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:48:54.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691842 2026-02-17T06:48:54.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691842 2026-02-17T06:48:54.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-120259084297 2-171798691842' 2026-02-17T06:48:54.246 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:54.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836499 2026-02-17T06:48:54.247 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:54.247 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:48:54.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836499 2026-02-17T06:48:54.248 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:54.249 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836499 2026-02-17T06:48:54.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-17T06:48:54.250 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836499 2026-02-17T06:48:54.250 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:48:54.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-17T06:48:54.399 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:54.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-120259084297 2026-02-17T06:48:54.400 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:54.401 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:48:54.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-120259084297 2026-02-17T06:48:54.401 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:54.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084297 2026-02-17T06:48:54.403 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 120259084297' 2026-02-17T06:48:54.403 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 120259084297 2026-02-17T06:48:54.403 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:48:54.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084297 -lt 120259084297 2026-02-17T06:48:54.554 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:48:54.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-171798691842 2026-02-17T06:48:54.555 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:48:54.555 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:48:54.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-171798691842 2026-02-17T06:48:54.556 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:48:54.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691842 2026-02-17T06:48:54.558 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 171798691842' 2026-02-17T06:48:54.558 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 171798691842 2026-02-17T06:48:54.558 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:48:54.706 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-17T06:48:54.707 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:48:54.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:54.708 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:54.887 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:55.033 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:48:55.033 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:55.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:55.034 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:48:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T06:48:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T06:48:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T06:48:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:48:55.210 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T06:48:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:48:55.311 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:55.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:55.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:55.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:55.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:55.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:55.313 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:55.462 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:48:55.462 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:55.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:55.463 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:55.639 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:48:55.640 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:48:55.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:48:55.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:48:55.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T06:48:55.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:48:55.820 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:56.022 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:56.169 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:48:56.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:56.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:56.170 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:48:56.349 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:48:56.529 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:48:56.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:48:56.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T06:48:56.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:48:56.530 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T06:48:56.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:48:56.930 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:56.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:56.930 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:56.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:56.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:56.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:56.931 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:57.084 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:48:57.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:57.085 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:57.086 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:57.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:48:57.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:48:57.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:48:57.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:48:57.266 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:48:57.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:48:57.267 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:48:57.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:48:57.267 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:48:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:48:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:48:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T06:48:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:48:57.450 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T06:48:58.250 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:48:58.251 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:48:58.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:48:58.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:48:58.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:48:58.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:48:58.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:48:58.252 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:48:58.405 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:48:58.405 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:48:58.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:48:58.406 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:48:58.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:48:58.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:48:58.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:48:58.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:48:58.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:48:58.590 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:48:58.591 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:48:58.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:48:58.591 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:48:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:48:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:48:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 13 )) 2026-02-17T06:48:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:48:58.779 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T06:49:00.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:49:00.379 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:49:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:49:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:49:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:49:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:49:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:49:00.380 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:49:00.535 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:49:00.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:49:00.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:49:00.535 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:49:00.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:49:00.720 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:68: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T06:49:00.721 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T06:49:00.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-17T06:49:00.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:49:00.722 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:49:00.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:49:00.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:49:00.722 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:49:02.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:49:02.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T06:49:02.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T06:49:02.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T06:49:02.030 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T06:49:02.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T06:49:02.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-17T06:49:02.031 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T06:49:03.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-17T06:49:03.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T06:49:03.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T06:49:03.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T06:49:03.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T06:49:03.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T06:49:03.202 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T06:49:03.203 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T06:49:03.204 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:49:03.204 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:49:03.205 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T06:49:03.206 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr:start osd.2 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=298407ee-1d38-432f-914f-f7fb62c52f3c --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648547/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T06:49:03.207 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T06:49:03.226 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:49:03.213+0000 7f6db9d608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:49:03.228 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:49:03.217+0000 7f6db9d608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:49:03.229 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:49:03.217+0000 7f6db9d608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T06:49:03.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T06:49:03.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T06:49:03.357 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T06:49:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T06:49:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T06:49:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:49:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T06:49:03.358 INFO:tasks.workunit.client.0.trial074.stderr:0 2026-02-17T06:49:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:49:03.358 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:49:03.513 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:49:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:49:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:49:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T06:49:04.515 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:49:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:49:04.515 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:49:04.668 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:49:04.716 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:49:04.705+0000 7f6db9d608c0 -1 Falling back to public interface 2026-02-17T06:49:05.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:49:05.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:49:05.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T06:49:05.670 INFO:tasks.workunit.client.0.trial074.stderr:2 2026-02-17T06:49:05.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:49:05.670 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:49:05.819 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:49:06.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:49:06.821 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:49:06.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T06:49:06.822 INFO:tasks.workunit.client.0.trial074.stderr:3 2026-02-17T06:49:06.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:49:06.822 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:49:06.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T06:49:07.697 INFO:tasks.workunit.client.0.trial074.stderr:2026-02-17T06:49:07.685+0000 7f6db9d608c0 -1 osd.2 41 log_to_monitors true 2026-02-17T06:49:07.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T06:49:07.975 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T06:49:07.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T06:49:07.976 INFO:tasks.workunit.client.0.trial074.stderr:4 2026-02-17T06:49:07.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T06:49:07.976 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:osd.2 up in weight 1 up_from 45 up_thru 0 down_at 42 last_clean_interval [40,41) [v2:127.0.0.1:6818/1217785724,v1:127.0.0.1:6819/1217785724] [v2:127.0.0.1:6820/1217785724,v1:127.0.0.1:6821/1217785724] exists,up 5eab9662-6bfd-45c5-a1e3-4a9c644734bd 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T06:49:08.124 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T06:49:08.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T06:49:08.125 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T06:49:08.125 INFO:tasks.workunit.client.0.trial074.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T06:49:08.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T06:49:08.125 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T06:49:08.126 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T06:49:08.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T06:49:08.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T06:49:08.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T06:49:08.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T06:49:08.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T06:49:08.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T06:49:08.384 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T06:49:08.384 INFO:tasks.workunit.client.0.trial074.stderr:1 2026-02-17T06:49:08.384 INFO:tasks.workunit.client.0.trial074.stderr:2' 2026-02-17T06:49:08.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T06:49:08.385 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:49:08.385 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T06:49:08.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836502 2026-02-17T06:49:08.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836502 2026-02-17T06:49:08.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836502' 2026-02-17T06:49:08.456 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:49:08.456 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T06:49:08.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084301 2026-02-17T06:49:08.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084301 2026-02-17T06:49:08.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836502 1-120259084301' 2026-02-17T06:49:08.526 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T06:49:08.527 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T06:49:08.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528322 2026-02-17T06:49:08.600 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528322 2026-02-17T06:49:08.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836502 1-120259084301 2-193273528322' 2026-02-17T06:49:08.601 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:49:08.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836502 2026-02-17T06:49:08.601 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:49:08.602 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T06:49:08.602 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836502 2026-02-17T06:49:08.603 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:49:08.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836502 2026-02-17T06:49:08.604 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836502' 2026-02-17T06:49:08.604 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.0 seq 21474836502 2026-02-17T06:49:08.604 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:49:08.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836502 2026-02-17T06:49:08.758 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:49:09.759 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T06:49:09.760 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:49:09.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836502 2026-02-17T06:49:09.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T06:49:10.906 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T06:49:10.907 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T06:49:11.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836503 -lt 21474836502 2026-02-17T06:49:11.061 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:49:11.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-120259084301 2026-02-17T06:49:11.061 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:49:11.063 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T06:49:11.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-120259084301 2026-02-17T06:49:11.064 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:49:11.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084301 2026-02-17T06:49:11.065 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 120259084301' 2026-02-17T06:49:11.065 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.1 seq 120259084301 2026-02-17T06:49:11.066 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T06:49:11.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084301 -lt 120259084301 2026-02-17T06:49:11.226 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T06:49:11.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-193273528322 2026-02-17T06:49:11.226 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T06:49:11.228 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T06:49:11.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-193273528322 2026-02-17T06:49:11.228 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T06:49:11.229 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528322 2026-02-17T06:49:11.230 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 193273528322' 2026-02-17T06:49:11.230 INFO:tasks.workunit.client.0.trial074.stderr:waiting osd.2 seq 193273528322 2026-02-17T06:49:11.230 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T06:49:11.391 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528322 -lt 193273528322 2026-02-17T06:49:11.391 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T06:49:11.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:49:11.392 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:49:11.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T06:49:11.575 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:49:11.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:49:11.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:49:11.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:49:11.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:49:11.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:49:11.576 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:49:11.729 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:49:11.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:49:11.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:49:11.730 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:49:11.914 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:49:11.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T06:49:11.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T06:49:11.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T06:49:11.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:49:11.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T06:49:12.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:49:12.016 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:49:12.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:49:12.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:49:12.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:49:12.016 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:49:12.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:49:12.017 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:49:12.163 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:49:12.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:49:12.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:49:12.163 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:49:12.345 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:49:12.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:49:12.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:49:12.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T06:49:12.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:49:12.527 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T06:49:12.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:49:12.728 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:49:12.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:49:12.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:49:12.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:49:12.728 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:49:12.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:49:12.729 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:49:12.890 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:49:12.890 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:49:12.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:49:12.891 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:49:13.072 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:49:13.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:49:13.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:49:13.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:49:13.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:49:13.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:49:13.073 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:49:13.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:49:13.074 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:49:13.255 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:49:13.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:49:13.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T06:49:13.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:49:13.256 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T06:49:13.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:49:13.656 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:49:13.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:49:13.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:49:13.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:49:13.656 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:49:13.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:49:13.657 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:49:13.807 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T06:49:13.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:49:13.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:49:13.808 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T06:49:13.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T06:49:13.989 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T06:49:14.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T06:49:14.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T06:49:14.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T06:49:14.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T06:49:14.172 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T06:49:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T06:49:14.973 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T06:49:14.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T06:49:14.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T06:49:14.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T06:49:14.973 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T06:49:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T06:49:14.974 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T06:49:15.122 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T06:49:15.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T06:49:15.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T06:49:15.123 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T06:49:15.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T06:49:15.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T06:49:15.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T06:49:15.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:70: TEST_recover_unexpected: sleep 5 2026-02-17T06:49:20.305 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:72: TEST_recover_unexpected: ceph pg repair 1.0 2026-02-17T06:49:20.445 INFO:tasks.workunit.client.0.trial074.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T06:49:20.454 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:74: TEST_recover_unexpected: sleep 10 2026-02-17T06:49:30.455 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:76: TEST_recover_unexpected: ceph log last 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.551820+0000 osd.1 (osd.1) 2 : cluster [ERR] 1.0 shard 0 1:602f83fe:::foo:1 : missing 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.551822+0000 osd.1 (osd.1) 3 : cluster [ERR] 1.0 shard 1 1:602f83fe:::foo:1 : missing 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.551913+0000 osd.1 (osd.1) 4 : cluster [ERR] repair 1.0 1:602f83fe:::foo:1 : is an unexpected clone 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.552034+0000 osd.1 (osd.1) 5 : cluster [ERR] 1.0 repair 1 missing, 0 inconsistent objects 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.552046+0000 osd.1 (osd.1) 6 : cluster [ERR] 1.0 repair 1 missing, 0 inconsistent objects 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.552069+0000 osd.1 (osd.1) 7 : cluster [ERR] 1.0 repair 3 errors, 2 fixed 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.557120+0000 osd.1 (osd.1) 9 : cluster [ERR] 1.0 shard 1 soid 1:602f83fe:::foo:1 : candidate size 0 info size 15806 mismatch 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.557170+0000 osd.1 (osd.1) 10 : cluster [ERR] 1.0 shard 0 soid 1:602f83fe:::foo:1 : data_digest 0xdfb728bf != data_digest 0xf85c6ade from auth oi 1:602f83fe:::foo:1(19'2 client.4174.0:1 dirty|data_digest s 15806 uv 1 dd f85c6ade alloc_hint [0 0 0]) 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.557172+0000 osd.1 (osd.1) 11 : cluster [ERR] 1.0 shard 1 soid 1:602f83fe:::foo:1 : data_digest 0xffffffff != data_digest 0xdfb728bf from shard 0, data_digest 0xffffffff != data_digest 0xf85c6ade from auth oi 1:602f83fe:::foo:1(19'2 client.4174.0:1 dirty|data_digest s 15806 uv 1 dd f85c6ade alloc_hint [0 0 0]), size 0 != size 15806 from auth oi 1:602f83fe:::foo:1(19'2 client.4174.0:1 dirty|data_digest s 15806 uv 1 dd f85c6ade alloc_hint [0 0 0]), size 0 != size 15806 from shard 0 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.557173+0000 osd.1 (osd.1) 12 : cluster [ERR] 1.0 soid 1:602f83fe:::foo:1 : data_digest 0xf85c6ade != data_digest 0xdfb728bf from shard 0 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.557241+0000 osd.1 (osd.1) 13 : cluster [ERR] deep-scrub 1.0 1:602f83fe:::foo:1 : is an unexpected clone 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.557363+0000 osd.1 (osd.1) 14 : cluster [ERR] 1.0 deep-scrub 0 missing, 1 inconsistent objects 2026-02-17T06:49:30.632 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:20.557365+0000 osd.1 (osd.1) 15 : cluster [ERR] 1.0 deep-scrub 4 errors 2026-02-17T06:49:30.633 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:24.379202+0000 mon.a (mon.0) 284 : cluster [ERR] Health check failed: 4 scrub errors (OSD_SCRUB_ERRORS) 2026-02-17T06:49:30.633 INFO:tasks.workunit.client.0.trial074.stdout:2026-02-17T06:49:24.379233+0000 mon.a (mon.0) 285 : cluster [ERR] Health check failed: Possible data damage: 1 pg inconsistent (PG_DAMAGED) 2026-02-17T06:49:30.640 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:79: TEST_recover_unexpected: timeout 60 ceph tell osd.0 version 2026-02-17T06:49:30.705 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:49:30.705 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19.2.3-705-g04cf6dd4", 2026-02-17T06:49:30.705 INFO:tasks.workunit.client.0.trial074.stdout: "release": "squid", 2026-02-17T06:49:30.705 INFO:tasks.workunit.client.0.trial074.stdout: "release_type": "stable" 2026-02-17T06:49:30.705 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:49:30.712 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:80: TEST_recover_unexpected: timeout 60 ceph tell osd.1 version 2026-02-17T06:49:30.773 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:49:30.773 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19.2.3-705-g04cf6dd4", 2026-02-17T06:49:30.773 INFO:tasks.workunit.client.0.trial074.stdout: "release": "squid", 2026-02-17T06:49:30.774 INFO:tasks.workunit.client.0.trial074.stdout: "release_type": "stable" 2026-02-17T06:49:30.774 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:49:30.780 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:81: TEST_recover_unexpected: timeout 60 ceph tell osd.2 version 2026-02-17T06:49:30.842 INFO:tasks.workunit.client.0.trial074.stdout:{ 2026-02-17T06:49:30.842 INFO:tasks.workunit.client.0.trial074.stdout: "version": "19.2.3-705-g04cf6dd4", 2026-02-17T06:49:30.842 INFO:tasks.workunit.client.0.trial074.stdout: "release": "squid", 2026-02-17T06:49:30.843 INFO:tasks.workunit.client.0.trial074.stdout: "release_type": "stable" 2026-02-17T06:49:30.843 INFO:tasks.workunit.client.0.trial074.stdout:} 2026-02-17T06:49:30.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:35: run: teardown td/osd-unexpected-clone 2026-02-17T06:49:30.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-unexpected-clone 2026-02-17T06:49:30.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T06:49:30.850 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-17T06:49:30.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:49:30.850 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:49:30.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:49:30.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:49:30.851 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:49:30.959 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:49:30.960 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:49:30.961 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:49:30.961 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:49:30.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:49:30.962 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:49:30.963 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:49:30.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:49:30.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:49:30.964 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:49:30.965 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:49:30.966 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:49:30.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T06:49:30.967 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-unexpected-clone 2026-02-17T06:49:30.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:49:30.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:49:30.981 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:49:30.981 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.648547 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-unexpected-clone 0 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-unexpected-clone 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T06:49:30.983 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T06:49:30.986 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T06:49:30.986 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T06:49:30.987 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T06:49:30.987 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T06:49:30.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T06:49:30.988 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T06:49:30.988 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T06:49:30.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:49:30.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T06:49:30.990 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T06:49:30.990 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T06:49:30.991 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:49:30.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T06:49:30.992 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-unexpected-clone 2026-02-17T06:49:30.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T06:49:30.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T06:49:30.993 INFO:tasks.workunit.client.0.trial074.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648547 2026-02-17T06:49:30.994 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.648547 2026-02-17T06:49:30.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T06:49:30.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T06:49:30.995 INFO:tasks.workunit.client.0.trial074.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T06:49:30.995 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T06:49:30.995 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T06:49:31.003 INFO:tasks.workunit:Stopping ['scrub'] on client.0... 2026-02-17T06:49:31.003 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-17T06:49:31.350 DEBUG:teuthology.parallel:result is None 2026-02-17T06:49:31.350 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T06:49:31.359 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T06:49:31.359 DEBUG:teuthology.orchestra.run.trial074:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T06:49:31.405 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T06:49:31.405 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T06:49:31.417 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T06:49:31.418 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T06:49:31.526 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T06:49:31.538 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T06:49:31.538 DEBUG:teuthology.orchestra.run.trial074:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T06:49:31.803 INFO:teuthology.orchestra.run.trial074.stdout: remote refid st t when poll reach delay offset jitter 2026-02-17T06:49:31.803 INFO:teuthology.orchestra.run.trial074.stdout:============================================================================== 2026-02-17T06:49:31.803 INFO:teuthology.orchestra.run.trial074.stdout:-23.186.168.132 132.163.97.4 2 u 23 128 377 69.656 +3.865 4.780 2026-02-17T06:49:31.804 INFO:teuthology.orchestra.run.trial074.stdout:*ntp1.wiktel.com .PPS. 1 u 85 128 377 38.756 +6.842 1.175 2026-02-17T06:49:31.804 INFO:teuthology.orchestra.run.trial074.stdout:+ntp.netlinkify. 214.176.184.39 2 u 11 128 377 13.284 +5.096 0.573 2026-02-17T06:49:31.804 INFO:teuthology.orchestra.run.trial074.stdout:+204.2.134.173 22.42.17.250 3 u 22 128 377 68.366 +4.781 2.071 2026-02-17T06:49:31.804 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T06:49:31.816 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T06:49:31.816 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T06:49:31.829 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T06:49:31.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T06:49:31.855 INFO:teuthology.task.internal:Duration was 4759.467732 seconds 2026-02-17T06:49:31.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T06:49:31.868 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T06:49:31.869 DEBUG:teuthology.orchestra.run.trial074:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T06:49:31.894 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T06:49:31.894 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial074.front.sepia.ceph.com 2026-02-17T06:49:31.895 DEBUG:teuthology.orchestra.run.trial074:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T06:49:31.944 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T06:49:31.944 DEBUG:teuthology.orchestra.run.trial074:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:49:32.137 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T06:49:32.137 DEBUG:teuthology.orchestra.run.trial074:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T06:49:32.146 INFO:teuthology.orchestra.run.trial074.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T06:49:32.148 INFO:teuthology.orchestra.run.trial074.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T06:49:32.148 INFO:teuthology.orchestra.run.trial074.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T06:49:32.148 INFO:teuthology.orchestra.run.trial074.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T06:49:32.148 INFO:teuthology.orchestra.run.trial074.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T06:49:32.193 INFO:teuthology.orchestra.run.trial074.stderr: 81.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T06:49:32.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T06:49:32.208 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T06:49:32.209 DEBUG:teuthology.orchestra.run.trial074:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T06:49:32.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T06:49:32.265 DEBUG:teuthology.orchestra.run.trial074:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:49:32.298 INFO:teuthology.orchestra.run.trial074.stdout:kernel.core_pattern = core 2026-02-17T06:49:32.307 DEBUG:teuthology.orchestra.run.trial074:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T06:49:32.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T06:49:32.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T06:49:32.370 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T06:49:32.374 DEBUG:teuthology.misc:Transferring archived files from trial074:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:03:08-rados-wip-bharath19-testing-2026-02-13-0903-squid-distro-default-trial/51019/remote/trial074 2026-02-17T06:49:32.376 DEBUG:teuthology.orchestra.run.trial074:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T06:49:32.433 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T06:49:32.434 DEBUG:teuthology.orchestra.run.trial074:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T06:49:32.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T06:49:32.476 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T06:49:32.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T06:49:32.492 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T06:49:32.492 DEBUG:teuthology.orchestra.run.trial074:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T06:49:32.510 INFO:teuthology.orchestra.run.trial074.stdout: 1180220 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 06:49 /home/ubuntu/cephtest 2026-02-17T06:49:32.511 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T06:49:32.528 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T06:49:32.565 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} duration: 4759.467731952667 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-17T06:49:32.565 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T06:49:32.682 INFO:teuthology.run:pass